RAISE Summit 2025 | Jonathan Ross, Groq

Clips
News
More from RAISE Summit 2025

Jonathan Ross

CEO, Co-Founder

groq

12 signs AI infrastructure is reshaping the IPO market: theCUBE’s RAISE Summit insights

Capital markets are being reshaped by artificial intelligence, with AI infrastructure emerging as a critical pillar of how companies scale, compete and go public. What was once considered a back-end concern is now front and center in IPO roadshows and investor decks.TheCUBE’s John Furrier talks about AI infrastructure at RAISE Summit.The shift is about more than hype. Investors are scrutinizing how companies embed AI into their business models — not just to drive innovation, but to ensure long-term operational resilience. It’s no longer enough to show top-line growth; firms need to demonstrate that their AI capabilities can sustain profitability and differentiate them in crowded markets

play_circle_outline Coverage of RAISE Summit 2025 focusing on AI leaders and infrastructure discussions.

play_circle_outline Jonathan Ross, CEO of Grok, discusses company’s identity as chip and systems provider.

play_circle_outline Balancing Geopolitical Challenges: Evaluating Compute Security and Energy Needs in Grok's Rapid Deployment of Resources

play_circle_outline Product-led growth strategy; emphasis on product quality over heavy marketing.

play_circle_outline European entrepreneurship gaining momentum; shift from legislative focus to fostering innovation.

Info
Transcript

Jonathan Ross, Groq

Jonathan Ross

CEO, Co-Founder groq

In this interview from RAISE Summit 2025, Groq founder and CEO, Jonathan Ross, joins theCUBE’s John Furrier to unpack what happens when demand for AI inference collides with global compute shortages. Ross shares fresh milestones, including 1.8 million developers on GroqCloud, 20 million tokens per second already online, a 20 000-chip sovereign cluster in Saudi Arabia that went live in just 51 days, and a brand new Helsinki data center that moved from agreement to production in a month. He explains why inference-only silicon is now the fastest, lowest-cost pat... Read more

explore Keep Exploring

What is the context or main focus of the RAISE Summit 2025 being covered by theCUBE? add

What type of company does Grok consider themselves to be, and what services do they offer? add

What is the current state of compute security and its importance compared to energy security, particularly in relation to the supply chain for GPUs and the timeline for acquiring compute resources? add

What is the relationship between product quality and marketing effectiveness in the context of product-led growth? add

What were the main topics discussed at the last RAISE and the AI Action Summit, and how do they compare to the conversations at the current event? add

bolt Powered by CUBE AI

Jonathan Ross, Groq

search

>> Welcome back, everyone, to theCUBE's live coverage here in Paris, France, for exclusive coverage of RAISE Summit 2025. I'm John Furrier, host of theCUBE. It's an international gathering of the leaders in AI, AI infrastructure, and of course entrepreneurs and enterprise leaders coming together to figure out how to harvest the value being created out of all their data and AI. Johnathan Ross, the CEO of Grok here, one of the fastest growing, if not fastest growing chip companies, systems companies. What do you call yourself, Jon? The chip company or systems company?

Jonathan Ross

>> We try to avoid labels. - Okay.

Jonathan Ross

>> We do token as a service, but we also do sell hardware.

>> Yeah. Great success. Congratulations.

>> Last time we talked two years ago at Supercomputing, I think, '24. You guys were just getting it out there. The numbers were off the charts. You had nailed the speed advantage. I think you were the first one in the industry to quote inference is the killer app.

Jonathan Ross

>> Yeah. - Okay. So you knew that was here.

>> Now that's everyone's line.

>> Yeah, we've been doing inference since 2016

Jonathan Ross

>> back before it was popular.

>> Talk about where you guys are at. So give a quick milestone update of Grok, and then first half of the year, stellar performance, expansion. Give us the quick stats.

Jonathan Ross

>> So we're at 1.8 million developers who've signed up. We have almost 20 million tokens per second of capacity deployed. In addition to the big sovereign deal that we did in Saudi Arabia. We've also just done an exclusive deal in Canada with Bell Canada to do the sovereign cloud for Canada. And there's a couple more in the pipeline.

>> Well- - And actually, I almost forgot.

Jonathan Ross

>> The reason that I'm here in Paris, we just announced this week our Helsinki data center, our first data center in Europe. And actually, we decided to launch it a month ago, a month and about three days, and it's actually serving traffic now.

>> All right. Congratulations. Hey, follow the sun, as they say. Okay. The demand curve right now is off the charts. Two years ago we were like, "Okay, this is going to be hot. " If you just fast-forward from 2024 to now, you guys are on the steep edge of the S curve. What's it like from a demand standpoint? What are some of the requirements? What are you hearing from folks? You got certainly leaders and researchers wanting this. You've got governments wanting it. What are some of the things coming at you guys from a business perspective? Can you share some of the inbound and engagements?

Jonathan Ross

>> It's all a single theme. No one can get enough compute for their AI. So it's like back in the industrial revolution, if you wanted to run a piece of equipment, if you wanted to run a washing machine or whatever, you actually had to plug it in and get power, and there wasn't enough power maybe. But today everyone has an AI model they want to run and they do not have enough compute. So there's a lot of companies going around trying to cobble up compute amongst the hyperscalers that can't get enough. And then of course, that brings us to sovereigns. What's gotten sovereigns interested is they actually cannot get compute in their own countries. So most of the folks who are deploying the compute, the hyperscalers and others, they're deploying in some of the markets where the most money is spent. That isn't necessarily where the most demand is, where the most need is. And so we helped Saudi Arabia set up a cluster of about 20,000 chips. That was about 51 days from when we signed the agreement till it was up and serving traffic. I mentioned the Canada thing. And so we're working on some more deals like that because those countries, they want to have access to compute so they can run those AI models, and they can't get it.

>> Yes, they have the washing machine with no power. I was also behind the main stage this morning. You were out on stage. I was kind of behind. You were in reverse on the screen, but I heard some of your comments. You had mentioned something about the industrial revolution, how electrical power grids, and you were comparing the growth of not having power on a country basis and how the dynamics were, and you were comparing it to today. Can you imagine a time where you did things without power and you're kind of referring it there? Are these countries looking at their power ability and then saying, "Oh, I might have to change my geopolitical alliance," or is it changing the dynamics politically? Have you had any of those conversations?

Jonathan Ross

>> So compute security is becoming as important as energy security. I think Europe had a real wake up with energy security, and I think the same thing's happening with compute. One of the problems with compute though is that when you're ordering GPUs, it typically takes 24 months. I think when Jensen was on stage at GTC, he explicitly said, if you want compute in two years, the day to sign the PO is today. And so in our case, we're actually able to do that in about six months because we have a much simpler supply chain. And so if you want to catch up in AI, right now, you have to think in months, not years.

>> Yeah. And that's their supply constraint. So what does that look like? I mean, take me through that because that's attractive. I have needs. What do I do? I sign a PO. What's my bill of materials look like? What's the system look like? How does it work? Explain it to me.

Jonathan Ross

>> We do build, operate, transfer. For example, in Saudi Arabia, we didn't just hand them a bunch of hardware. We set up the data center, we operate it, we man it, we have people doing all the servicing, we run the software, we handle the whole thing. Same is going to be true in Canada. We're going to be doing that. For some other customers, they may actually want to take it after we've gotten it set up for them and it's humming like a smooth purring machine, they may actually want to take ownership of that later. Others may want to take it on themselves from the beginning if they're very sophisticated. But most people, they want GPUs, they want AI compute, they just don't know how to operate it or what to do with it afterwards, and we're pretty good at handling the whole thing for them.

>> So you're saying that you're the alternative for the lag on GPUs, you can deliver the performance they need, and do it faster.

Jonathan Ross

>> And we focus on inference. So we can't help you with training. For that you should still use GPUs. GPUs are really good at training. But when it comes to inference, we are faster, we're less expensive, and we use less energy. We use about a third as much energy per token produced.

>> What's some of the stats on the token output, first token out multi-step reasoning?

Jonathan Ross

>> So on some of the larger 400 billion parameter models, we're doing about 600 tokens per second, but that's our slowest speed. We do that to be economical. We could run faster. Right now there's more demand than there is capacity. So we've optimized for throughput.

>> Talk about the Grok brand, because one of the things when you're moving in fast, and you guys are, you got to build the brand, get the customers, and you got to have the demand gens, the marketing golden triangles they call it. How do you balance that? Because you guys are growing fast. What's your go-to-market look like right now? What are you optimizing for? What's your focus on that piece?

Jonathan Ross

>> Product-led growth. Actually, our CMO has a saying she likes, which is the fastest way to kill a bad product is good marketing, which is if your product can't hold up and you start marketing, it's going to die very quickly. In our case, we have a really good product, so our goal is to get it out there as quickly as we can and let people just play with it. And so we've been shocked. I've been in meetings with C-level executives. There was one where I was meeting with a CTO and he's like, "This pitch is so amazing, we have to try it. " Turns to the guy sitting next to him. He's like, "Can you get on Grok and start benchmarking tonight? " And he just turned back to him and said, "Grok's my default, I use it for everything."

>> So he's already there.

Jonathan Ross

>> And that's where product-led growth comes in. So we've had that 1.8 million developers who've signed up. So pretty much everywhere we go, people have tried us, they've experimented with us, and they already know how to use us. So then all we have to do is come in at the top and meet that with the bottom, who all want to use us.

>> Got it. You're surrounding the castle, as they say. Scope the customer needs. Is there a certain profile customer that you go to? Where does the line stop? Developers can come up and play with it. We know that. Is there a profile floor for you in terms of customer needs? I guess what's the zone of size of the customer you guys want to go after?

Jonathan Ross

>> So we can handle pretty much any customer, but what has been the easiest for us to onboard is actually more sophisticated customers. So what ends up happening is a less sophisticated customer may not realize how important the speed is or the low cost, and then all of a sudden they start getting the bills and they're like, "Gosh, this is starting to get expensive. " The more sophisticated ones, what we've noticed is their spend on token as a service, and we're token as a service, and I'll explain that a little more, almost exactly equals their revenue. And every time you give them a lower cost, they actually end up consuming more tokens. They don't actually just spend less. They consume more. And the reason is unlike normal software, historical software where you write it and it's the quality that it is, with generative AI, if you turn up more tokens, if you turn up the reasoning, if you do more in parallel, you can actually make the product better and smarter. And so everyone's competing on the quality, and so the cost and the speed allow you to dial up the number of tokens and improve the quality of the product. So our ideal customers are the ones who've discovered they need more compute, and they can't get enough and they need it faster and they need it cheaper, and those are the ones that just sort of pull us.

>> So the ones that have figured out it's not the one-

>> shot token, it's really the more reasoning. Because the demand for tokens with reasoning is higher.

Jonathan Ross

>> Well, yeah, and I'll give an example. So one of our engineers was doing a bunch of formal theorem proving because we do chips and we want to get them right. And so he would use Claude Opus, which would get the answer in the smallest number of iterations, on average two. But then when we would take a small open source reasoning model on our hardware, something like Qwen3-32B, it would always solve it faster, even though it took more iterations, was so many multiples faster than a GPU. The amount of time is less, but the cost is also less. So the question is-

>> That's counterintuitive because you'd think more tokens, more cost.

Jonathan Ross

>> But if it's a less expensive model-

>> Yeah, that's what I'm saying. - ...

Jonathan Ross

>> then even though it's more tokens, the cost is lower.

>> Yeah, I was just talking with Matt Hicks from Red Hat. I asked Matt because he's a Linux guy. I'm like, "You're the colonel, you want to have a very efficient colonel. " And he's said, his goal is he's not playing at the world you're at, he's more in software, but his philosophy was they're just optimizing for the lowest cost token, period. Because there's going to be more of them coming with agents. And so his point was, if this doesn't drop, if we don't make this as cheap as possible, the first call... Because he was kind of going down the same thread you were, which is you don't know how many calls because agents are going to have thousands of calls, right?

>> Well, in our case, so what most

Jonathan Ross

>> of the customers are trying to do is they'll have a budget, a certain number of tokens that they want to expend and an amount of time, and they want to do the maximum that they can in that amount of time to get the quality. So by bringing the cost down, they get more tokens. But also in our case, because we own the full vertical stack, and we got a shout-out from Mark Zuckerberg actually on Ben Thompson's Stratechery, where he said he loved how vertically integrated we were, that vertical integration brings the cost down. So actually, our products very often are CapEx, OpEx, profit, everything combined is actually less than just the OpEx to run a GPU. So you could literally give someone a GPU for free and it would be more expensive to generate tokens out of that than buying our product and deploying it.

>> Yeah. And I think- - For inference only, to be specific.

>> First of all, I love inference only

Jonathan Ross

>> because that's where the killer app is there.

>> You had called that, right? We agreed of course on that one. It's obvious to us. But I think your point about the token costs on not the fake... I won't say fake. The false results of Opus, your point there was it looks better because it's two-

Jonathan Ross

>> It's two . - ...

>> but it's faster here.

Jonathan Ross

>> But if I can do five iterations, but I do it in less time than I did the two, it's faster, it's cheaper, what does it matter?

>> Yeah. And my point there, and I'd love to get your reaction, is that we're seeing that the agentic side is going to be a thousand X plus tokens. So that kind of workflow is a predictive nature of what agents are going to do. So to your example of the two to faster time, that's just going to multiply with agentic, right? I mean, is that-

Jonathan Ross

>> 100%. And in agentic, you also have this thing where you want to do a bunch of things in parallel too, so you got to get the cost down because you want to try multiple things. So the cost is pretty extremely-

>> Cost is a huge deal.

>> Yeah. - All right. So, okay, you've got the cost leader

Jonathan Ross

>> and high performance leader.

>> That's, again, a winning product. So what's the marketing strategy?

Jonathan Ross

>> Put it out there and let people try it and then they tell other people about it. It's worked. We've gone viral.

>> Yeah. Check it out, everyone. Grok is the answer. I think this is a huge issue because we saw that in CloudGain FinOps, efficiency is everything. Speed is everything. Everyone that comes on theCUBE these days past year, the number one thing is speed is the new competitive advantage. You got to go fast both on technical performance but also business model. So from a company, I got to get my product to go faster. So that seems to be the biggest theme that's consistent.

>> I'll give you a great example.

Jonathan Ross

>> So we had a customer come in the other day here at this event and we were chatting with them. At first we thought they wanted to bring the cost down. And they were sort of waffling on a little bit and they're like, "Look, I haven't ramped it up yet. Cost is going to become a big deal in the near future. Right now it's not the problem. " And we're like, "Okay, but if it was fast, would you get more engagement and get more users? " And he's like, "Absolutely. Oh yeah, absolutely."

>> Yeah, that too. We got that too.

>> So then all of a sudden the conversation went to speed,

Jonathan Ross

>> and then when you get speed, you get more users, then the cost matters. So we solved it in advance.

>> All right. So here's a question for you.

>> I know you talk to a lot of customers, you're traveling a lot, doing a lot of engagement with folks. What is your view of when that whole agent reality kicks in? Not agents as a product, but the notion that the token amounts are coming so fast and volume that to run through those tokens, process those tokens or whatever word you call it-

Jonathan Ross

>> It's a capacity limitation. So what we are seeing is a lot of these folks who've nailed it on the agent side, and not everyone has, but those who their business is like hockey-sticking, they're seeing 20 to 30% growth per month. Think-

>> In tokens or...

Jonathan Ross

>> In revenue and tokens because they're almost one-to-one.

>> And so when you are growing 20 to 30% per month and you're already capacity limited, things that you had to plan for for two years are never going to be able to catch up.

>> So it's a knife edge for them or a flat line. They pick peak out at capacity-

Jonathan Ross

>> Well, either they just have to give up on adding more customers or they have to find a way to add capacity that can be grown much more quickly.

>> Okay. So how do you help them? Same model.

>> They call you up and say-

>> Same model.

Jonathan Ross

>> So they call us up, they often hand us their model, we compile it, we show them performance, they test it, and they're like, "Great, this is thumbs up," and then we deploy it.

>> All right, Jon, it's always great to talk to you. I'd love to do a deeper dive. I know you're super busy, but I have to ask you about the conference. Again, I saw you on stage. I know you got tons of meetings, your schedule's been booked. Okay. Back to the good times with theCUBE. What are some of the hot conversations you've had here? What's your takeaway here in Paris besides the great city, but hallway meetings, what's your takeaway? What's the vibe? Obviously entrepreneurship. That was my surprise. A lot more entrepreneurial activity than I thought.

Jonathan Ross

>> Let's nail that one for a second. So I was here for the last RAISE and then I was also here for the AI Action Summit. And in those two events, all of the conversations were about legislation, how do we stop this AI, how do we protect ourselves?

>> . - And this time,

Jonathan Ross

>> you're right, it's all entrepreneur.

>> And so I actually think Europe's catching on and doing this and moving very quickly. And there's been these announcements where they're going to be building some of these gigafactories. And I think Europe is going to get in the game and the US and Europe are going to be able to work closely.

>> I think there's a revolution going on in Europe

>> and I think that's a great point. Even some of the European... I interviewed Daniel Dines, founder of UiPath, who's Romanian, but he's proud of his European heritage. I asked him at the last question, just kind of end it on a nice note, "Hey, give us your advice and what would you share to other entrepreneurs? And just in general European... " He was preaching to the European... He was basically shaming. That's my word. That's my takeaway. But he was very adamant about, "Look it, we have to be leaders and if we rearrange the deck chairs, the world's going to . Because you've got China, the Middle East, and the US as powerhouse, and in the Middle East, they just want their sovereignty, but I think China, America... " So I think Europe has a time, a moment in time right now where if there's not a grassroots revolution from the entrepreneur and the business community, they could be stuck in legislative gridlock. I mean, the consequences of that are mind-blowing because if Europe becomes irrelevant, I mean, there's tons of entrepreneurs in Europe. Tons of business. And a lot of it's on-prem.

Jonathan Ross

>> Yeah. - So it's good.

>> Well, so the way that I look at it,

Jonathan Ross

>> we are in a more select environment here at the summit. And so I would expect that as you go further out in Europe, you're not going to see the same sort of positive feelings. At least now at the summit, it's very positive. And last year that was not the case. And at the AI Action Summit, that wasn't the case as well. And so at least all of the people who are in the lead get it. It's about how do we enable it?

>> Yeah. How do we get people across the chasm?

>> One of the things at the theCUBE I've observed is that as we go global, we got the Palo Alto, New York Stock Exchange studio as an independent operator there, we're going to come to Europe. You're seeing entrepreneurship here. And the hackathon that was here-

>> Which we sponsored.

>> You guys sponsored. Shout-out to you.

Jonathan Ross

>> I mean, I love that hackathon. But what's noticeable about that hackathon is that for the first time in my career, I've really seen the first signs of Silicon Valley being syndicated. Because now we're at a level now where the valley mindset of entrepreneurship is not just Silicon Valley. I mean, this has been talked about before. It was trying to replicate Silicon Valley, but that hackathon is being won by people who aren't even here. So you have people, teams on-site and remote all around the world, and they're winning. I was talking to another entrepreneur who sold his company for $80 million. He did it in six months.

Jonathan Ross

>> Wow. - Okay. So this is the kind

>> of electronics on limited scale that's coming that's being enabled. So what you're doing, what the industry's doing at the infrastructure enablement, I think is going to be a completely different scale game. And things like six months of work, sell your company for $80 million, change education, just the benefits are just multifold. Are there any reaction to that from your standpoint of where you see... I mean, there's many examples, but what's the frothy area that's good, that's enabled that you see growing fast with that kind of democratization? What do you see, Jonathan, that's encouraging you?

Jonathan Ross

>> Well, I always give a shout-out to STATION F every time I'm in France. I think one of the bigger problems in cultures that aren't used to sort of an entrepreneurial bend is that if you have a very entrepreneurial person and they're surrounded by a bunch of people who are saying, "You can't do that, you should quit. Why don't you get a real job?"

>> "You failed. You're a loser."

Jonathan Ross

>> Yeah, it'll wear on them and eventually they'll stop.

>> But when you surround someone with 3,000 other people who are risk on, then they're risk on and they're going to go and they're going to build a company. And so we chat with a bunch of the startups from STATION F and they're all risk on, they want to do things. And so my philosophy is when you catch someone doing something right, let them know.

>> RAISE. - So STATION F is amazing.

Jonathan Ross

>> I think RAISE is amazing. The hackathon's amazing. And also there's a VC culture that's starting to grow in Europe, and it's a little more... The VC culture in Europe used to be a little more old-school with very onerous terms. Some of the newer VCs are more like the Silicon Valley VCs, which are like, how do I enable the founder as opposed to how do I maximize my control over the entrepreneur? So that kind of stuff's helpful. It's a bit of a mindset shift. But also, I do think that in Europe they want to do this, and believing that they can is the first step. And so just to point out, we decided that we were going to build this data center in Helsinki or set it up, and then fill it. A month and roughly three days later, it's up and serving traffic. That happened in Europe. Yes, we are who we are and we build fast and we move quickly, but Europe didn't get in the way. So now can Europe start to pull projects forward at that speed and start delivering multiple of them?

>> You know what they say? One penguin jumps in the water, they all jump in. Let's hope Europe follows that. Jonathan, always great to see you.

>> Let's do the next data center in a month and two days.

Jonathan Ross

>> Okay. Yeah, I want to be there for the grand opening.

>> Jonathan Ross, entrepreneur, CEO, founder of Grok. This is what it's all about, moving fast. Surround yourself with people who are going to get to that next level of innovation. Again, the enablement coming is off the charts. Of course, theCUBE's bringing all the action here in Paris. Thanks for watching.