theCUBE + NYSE Wired: AI Factories - Data Centers of the Future | The Inference Engine: Building AI That Performs at Scale

The Inference Engine: Building AI That Performs at Scale

In this interview from theCUBE + NYSE Wired: AI Factories – Data Centers of the Future event, Glean co-founder and CEO Arvind Jain joins theCUBE’s John Furrier to unpack what’s really working in enterprise AI today and what comes next. Jain explains why knowledge access remains the first successful AI use case at scale and how Glean’s enterprise search brings AI into everyday work. He details the past year’s lessons with AI agents – from the need for guardrails, security, evaluation and monitoring to democratizing agent building so business owners (not just data scientists) can create production-grade agents. The conversation dives into Glean’s vision of the enterprise brain powered by an enterprise graph, highlighting the importance of deep context, human workflows and behavior to reduce “noise” and drive outcomes. Jain outlines core building blocks – hundreds of enterprise integrations and a growing actions library – that let agents securely read company knowledge and take actions across systems (e.g., CRM updates, HR tasks, calendar checks). He discusses how organizations are standing up AI Centers of Excellence, prioritizing “top 10–20” agents across functions like engineering, support and sales, and why a horizontal AI data platform that unifies structured and unstructured data – accessed conversationally and stitched together via standards like MCP – sets the foundation for AI factory-scale operations. Looking ahead, Jain says Glean’s upgraded assistant is evolving from reactive tool to proactive companion that anticipates tasks and accelerates productivity.

Share this session

Clips
More from theCUBE + NYSE Wired: AI Factories - Data Centers of the Future

Felix Ejeckam

Co-Founder & CEO

Akash Systems

Mitesh Agrawal

CEO

Positron AI

Sam Awrabi

Founder & GP

Banyan Ventures

Sid Sheth

CEO

d-Matrix

play_circle_outline Beyond Training: Inference-Centric AI — Diverse Deployments, Models, Metrics, and Heterogeneous Hardware

play_circle_outline NVIDIA dominance and Groq deal validating inference market demand

play_circle_outline Inference spending growth: Anthropic ARR rise indicates surge in token-based revenue

play_circle_outline Investors focusing on AI-native infrastructure and vertical AI applications

play_circle_outline d-Matrix Avoids HBM Dependency Amid CoWoS Constraints, Leveraging SRAM, Custom DRAM and LPDDR

play_circle_outline Chiplets Power Intelligence Shift: Scaling Core-to-Edge and Cell-Tower Compute

play_circle_outline Model developments: longer context lengths and world-simulation models needed without degradation

Info
Transcript

The Inference Engine: Building AI That Performs at Scale

Felix Ejeckam

Co-Founder & CEO Akash Systems

Mitesh Agrawal

CEO Positron AI

Sam Awrabi

Founder & GP Banyan Ventures

Sid Sheth

CEO d-Matrix

In this interview from theCUBE + NYSE Wired: AI Factories — Data Centers of the Future, Sid Sheth, chief executive officer of d-Matrix, joins Mitesh Agrawal of Positron, Sam Awrabi, founder and general partner at Banyan Ventures, and Felix Ejeckam, chief executive officer of Akash Systems, to talk with theCUBE's Dave Vellante about why inference has become the defining battleground for AI compute. The panel explores how the NVIDIA-Groq acquisition validated inference as a multibillion-dollar market and why no single architecture can serve the full range of wo... Read more

explore Keep Exploring

How has the AI computing landscape changed since November, and what does the growing focus on inference (including effects on deployment, platform requirements, and market winners) imply for different customers and use cases? add

What challenges arise from the wide variety of inference workload requirements (are workloads all "snowflakes"), and how should those differences be handled? add

How has Anthropic's rapid ARR growth affected inference compute requirements, and what challenges and optimizations will the AI industry face moving forward? add

What is your investment focus in AI, and why are you investing in companies like Positron and Akash (particularly regarding AI-native infrastructure and inference power efficiency)? add

How has NVIDIA's dominance of the GPU/HBM ecosystem influenced your hardware design choices—particularly your decisions about packaging (CoWoS) and memory types (HBM, SRAM, custom DRAM, LPDDR)? add

How do you expect intelligence (AI) to diffuse from the core/data centers to the edge, and how is your company's platform designed to support that transition? add

What aspects of model development — for example reasoning, continual learning, increasing context length without degradation, or world-simulation/alternative transformer architectures — are you most excited about or see as most important for the future? add

bolt Powered by CUBE AI

The Inference Engine: Building AI That Performs at Scale

search

Dave Vellante

>> Hi everybody. Welcome back to our Palo Alto studio, theCube and NYSE Wired GTC pregame, the whole industry, the tech industry is descending in Silicon Valley in San Jose, getting ready for the big Super Bowl of the tech industry. And we're going to talk about inference, inference at the edge, building platforms that scale. Everybody talks about training. We're going to talk a little bit more about inference and what's actually happening. We're just back from Mobile World Congress and that's all everybody wanted to talk about. With me to my left is Sid. He's the CEO at d-Matrix. Sid good to see you again. You've been on a couple of times and we're going to get the update. As well as Mitesh, you were here a year ago. Mitesh is with Positron. And Sam, thanks for coming in, Sam. He's the founder and general partner at Banyan Ventures. You're doing some cool investments in companies like Felix's firm. He's the CEO of Akash Systems. So thanks guys for coming in.

Sid Sheth

>> Thanks for having me.

Dave Vellante

>> So of course, when we were here last year having this conversation, we've had this conversation at our NYSE studios. Well, everybody is talking about training. It's where all the activity is. Tons of CapEx going in, but sort of the hidden secret here, the gem is what's going on elsewhere, inference. And you guys are doing some cool stuff all around this table. Give us the update from last time we talked. What's new in the market? What's real? What's hype?

Sid Sheth

>> Yeah. So first of all, lots has changed and it's great to be back with you. Love doing these. It's just a lot of fun. I think we spoke in November.

Dave Vellante

>> Yeah.

Sid Sheth

>> And my God, the world has changed dramatically since November. A few things happened. One is I think the little secret that you talk about an inference, I don't think it's that much of a secret anymore. I think everyone across the world now knows, especially after the NVIDIA Groq deal that happened December. There is clear acknowledgement that the next big wave of AI computing is going to be around inference. So I think that secret is pretty much out of the bag at this point. And I think people are just trying to figure out what does that really look like because it's not a one size fits all. This is something we've been saying for a while. You can't really badge a single computing platform for all of inference because inference is done at different points in the network. You do it in big data centers, small data centers. We do it in edge applications. It's just going to really spread out. And there's big models, small models, there is different metrics of success. How you really ... Every customer cares about something that's different and unique to them, depending on the applications that they're running. So very difficult to kind of slot one solution into everybody's needs. So that's the beauty of the space. I think there's going to be multiple winners. And at this point, I think everyone is deploying compute, not just for training. Training is going to continue, but there's lots of deployments and specific announcements that have been happening literally in the last three months around inference capacity and inference deployments. And that to me is very exciting. I think we've all been waiting for this moment, 2025 was that moment when we transitioned away from training only to a training plus inference world. And I think 2026 is going to be all about huge announcements around deploying inference at scale and people trying to build new applications to take advantage of all of that.

Dave Vellante

>> So Mitesh, what kind of challenges does that create? Because as Sid was just explaining, it's a lot of different requirements. Is everything a snowflake? How do you deal with that? Of course, he mentioned the Groq deal, Groq with a Q folks, not the other Grock with the K. So that's great news because that essentially set the market tens of billions. So that's good, confirmation, you're not running away because there's so much demand for what you do, but explain your perspective on all this.

Mitesh Agrawal

>> Yeah. I think to your point around is everything a Snowflake, the kind of answer is yes and no. The point there is that there is many inference applications and people really care about ... Different people care about whether it's time to first token or latency or time to last token or context. There's so many kind of variables that go into defining an inference workload. And I think what happens is people rightly so have been using NVIDIA GPUs for training. They're the best in class that is out there for training applications. And then, they have been running the inference on NVIDIA GPUs. And NVIDIA GPUs so far have been really the outperforming kind of silicon out there for ... especially when you're talking about inference on a dollar basis. But what's been proven out, whether it's companies like Cerebras and others, and then companies like Sid's d-Matrix, Positron AI, what we're coming into the market is like, well, you have many angles to approach this. Whether it's fast speeds, that's SRM based architectures that have come out, whether it's like high amount of context and memory requirement because the KV caches are exploding as people are using applications for code generation, video generation, so this huge amount of memory requirement. So all of these different angles means that as much as NVIDIA is at the forefront of technology, there is room and kind of spaces for new companies to come in and really target those applications. And really outshine or really perform on those. And that was kind of the validation around the Groq that you mentioned. That was the validation of that industry perspective. And on the inference side, what I'll say is the other actual challenge is to your point, and what Sid kind of said, until 2025, training was the majority of spending. We'll see what the numbers in 2026 outlined to be, but it's at least going to be kind of even Stevens. And you can just look at one aspect or one stat. Anthropic, literally 15 months ago was two billion ARR, now it's 20 billion ARR. That ARR is inference ARR, right? That's selling tokens. That's inference. So they have to add to ... even if you assume that they have some level of optimization in that revenue, it's anywhere between five to 10 times more compute that they've required within a 15-month period for inference. So whatever they were spending 15 months ago, they have like five to 10X of that, right? And that's kind of one of the biggest challenges for the industry moving forward is how do we balance or how do we figure out how to get more and more compute? And this is kind of where people that are different types of competition comes in because you can optimize on power setup, you can optimize on total throughput, you can optimize on speed of tokens and you can support different applications. And that's kind of the snowflake part of it. But it is going to be very interesting and it is actually the most exciting time to be in this industry for all of us.

Dave Vellante

>> Thank you for that. So Sam, as an investor ... I got two questions. Is SaaS dead and what are you investing in? It was kind of tongue in cheek, but I mean, the market just ... as Mitesh was saying, it just changed and Sid was saying it changed overnight. You have to allocate capital in that crazy market. So what's your thesis these days? What are you investing in?

Mitesh Agrawal

>> Yeah. So I'm very fortunate, both companies here, Positron, and Akash, I'm a founding seed investor in both of them and really proud of that. So for me, I focus on AI native infrastructure and AI native vertical application software. And I define that as building with state-of-the-art advantages natively in your product that push what's possible with AI from day one of launching your company. So it feels like some of these changes are overnight, but I think for the infrastructure side, it's been a long time coming. There's a shift happening in the market where we're going from a one size fits all generalization approach of training, fine-tuning and inferencing with the same hardware to, hey, now you can actually pick the type of inference hardware you would like to pick to get the best watt per dollar efficiency. So for example, with specifically inference in 2024, it was estimated for the compute spend of OpenAI. Only 30% of that in 2024 was actually for inference. Now go to 2025 and it's closer to 60 to 80% of their compute spend is now on inference. So we're going from a world of spending a lot of money on training to spending a lot of money on inference and that power cost, that watt per dollar efficiency of the inference hardware really, really matters because I think there's a misconception that the hardware costs, all the money or the data center costs a lot of money. But if you actually take a step back, it's anywhere between 60 and 70% of the total cost of owning the hardware, actually goes to the power cost. And now, we're seeing H100s could last eight to 10 years. This hardware is lasting a lot longer than people maybe initially thought. And so, it's the power cost that is very expensive over the cost of the hardware. And companies like d-Matrix, Positron, and what Akash is doing on the cooling technology really makes the power efficiency much better on a watt per dollar basis. And that's where I'm investing at the inception stage into these companies.

Dave Vellante

>> Interesting. And when Michael Berry came out with that sort of negative sentiment around the life of accelerators, I actually wrote a piece saying, I think people need to rethink this because I think the life is going to be a lot longer than folks think, I would say at least four to six years, and I think that's playing out. In fact, I think everybody was sort of conservative about that. And to your other point, just a finer point on this, so Jensen in this conference calls often says X percent of our business is inference. Now, of course that's inference. As you say, it's the same hardware, basically whether it was hoppers or Blackwells. And a lot of that, of course, was open AI on Azure, but you guys are all doing sort of specialized things and I want to come back to that. But before I do, Felix, you got to explain how you guys fit in with Akash, what you're doing with this really interesting synthetic diamond materials. How does it play in helping with this problem?

Felix Ejeckam

>> So I think what we just heard ... First of all, thanks for having me. Felix Ejeckam is my name, CEO of Akash Systems. What you've just heard from Mitesh and from d-Matrix is a tremendous amount of compute that's coming online and applying tremendous amount of pressure on the grid. There isn't enough power in the grid in the United States to support the kinds of compute that we're talking about here. And if you look at it around the world, it's even greater. I think that the US is deploying one tenth the amount of power that say China is deploying on any given year. So you look at that and say, "Well, what are we going to do if there's not enough power?" We come into the picture and say, "Well, hold up. You don't need to invest in 100 times more infrastructure to be able to support what the market needs with these new revolutionary technologies that are coming in online." And so, what we do at Akash is we've taken a passive invention, piece of lab-grown diamonds, and we use it to cool down very, very hot GPUs.

Dave Vellante

>> Can you hold that up?

Felix Ejeckam

>> Sure.

Dave Vellante

>> Do you guys get that?

Felix Ejeckam

>> This is what a lab-grown diamond looks like, a piece of material that we grow in the lab. And we take this and we apply it. It's a very thermally conductive material. In fact, it's the most thermally conductive on earth. We put this right on the GPU and it brings the temperature down by about 10 to 15 degrees Celsius. You don't have to install new infrastructure. You don't have to put any new equipment at play. You take your standard servers and we've partnered with MiTek. We've got some other partnerships. We actually did this for the first time with a super micro server. We also do this with an NVIDIA server and we apply it together with the server and release it to the market. We just announced release of a 350 intrinsic server with AMD just last week. And that is able to get people like d-Matrix and Mitesh's company online. And by the way, you can bring the PUE to closer to 1.0. He was just talking about, do we have enough power to keep with the dollar per watt curve? Well, PUE is about taking all of the power going into a data center and having most of it, if not all of it, go into compute rather than cooling. You can now do that with diamond cooling.

Dave Vellante

>> Okay. And so, at what part of the process do you lay that on top of the GPU? Does the OEM does that?

Felix Ejeckam

>> So we do it right here in the Bay Area. So we at a car systems, we grow this and we put this right on the hot chip, the GPU, and then the rest of the server looks the same. And then, we deliver that with our partners to customers. We have this with MiTek computing. We have another one with the 300 with Super Micro and we have others that are coming down the pike in the next few months.

Dave Vellante

>> I want to follow up on the comments you're making about power. John Furrier just wrote a piece, another preview piece to GTC. He talked about all these constraints. He mentioned fab constraints. So obviously, compute is a constraint. Power is a constraint. We've been working on the premise that our industry has to solve the power problem or we're going to lose the AI race. So you can brute force it, as you say. The hyperscalers are all building sort of their own power generation systems. So it seems to be at least going that direction. At the same time you hear, there's still backlash from citizens saying, "Not in my backyard." So you're getting a lot of that. So how much of a constraint ... I mean, obviously near term, we know it's a constraint, but we've been going in the assumption that you guys, the industry is going to solve that. Are we underestimating the industry's ability to solve this problem or do you align with that premise?

Felix Ejeckam

>> I'll comment on that and say that we actually support not building any new power supply. So you can take your existing power plants, your existing power capacity using our technology, and you would get more compute. You could literally turn off the air conditioners in your data centers if you're going to use diamond cooling and not have to get liquid cooling or any of those other technologies, which are fabulous by the way. But you can turn those off and you get more money back to you as an operator. Roughly, we estimate about a million dollars that you get per server operating with diamond cooling in your GPU.

Dave Vellante

>> So the economics are very favorable for token-

Felix Ejeckam

>> Correct. Correct.

Dave Vellante

>> The other constraint that John mentioned was, and you guys touched on this a little bit, was memory. How have the exorbitant increases in memory affected you? I was out in Utah a couple of weeks ago talking to some of the AI cloud companies. Six months ago, all they wanted was flash and they didn't want to touch any spinning disc. And today, they can't get enough spinning disk because it's significantly ... you see what's happening in the Seagate stock and Western Digital and so forth. How has that affected your business or is it just there's so much demand that it doesn't matter?

Sid Sheth

>> Let me touch upon that a little bit. Of course, at d-Matrix, we deliberately built a solution. We knew this is going to be an issue at some point in the future because you have one large GPU based platform that basically has put a stake in the ground in terms of how that solution is going to be built. And assuming that that solution is the predominant solution, it's very difficult for somebody else to kind of get attention from the ecosystem building something that's going to be similar, right? So everyone here on this panel is trying to do something that's not that, right? And as startups, we have to get creative. We have to hustle. We can't be building, that's a me too product like the big incumbent in the market. So all of us have taken different approaches. In our case, we said we are not going to use CoWos, which is the silicon interposer technology that packages HBM. We said we are not going to use HBM memory, right? So we have deliberately made a choice where we use SRAM, custom DRAM. I think we've talked about this on the show before, where we stack custom DRAM directly on top of compute. And then, we have another tier of memory which is LPDDR. And we are seeing some spillover effects already into that LPDDR market. The LPDDR market is already beginning to get constrained. Luckily we use a version of LPDDR, LPDDR5, which is not as constrained as the four, because four is getting end of life very quickly, but we are not as badly affected as anyone who might be using anything that fits into the NVIDIA ecosystem, right? Because that's where there's a real constraint.

Dave Vellante

>> Well, to your point, NVIDIA sucked up all the co-ops demand and 2025 is really problematic. It's easing up a little bit, but there's still a constraint in the fabs and now, it's shifting to the front end and TSM has to balance that and they're not going to build more CoWos capacity until they ... they can deal with the front end. And so, you have to find different ways. Mitesh, would you add anything to that memory discussion?

Mitesh Agrawal

>> I think Sid has answered everything. I think the one thing I'll just say to it is the prices of memory does affect the entire ecosystem all the way from HBM to DRAM, including LP5X. Now, to Sid's point, LP5X much more mobile memory. Actually, the mobile market is not doing that well. So it allows for a better allocation or availability of memory, but the prices have gone up. I think just ourselves looking at the prices from literally 12 to 14 months ago to nine, and this is not spot ... spot market is crazy, right? But even the contractual market, that's gone up a lot, anywhere between 50, 60 to 100 points of price increase in the contracture and memory pricing. So it does flow through, but I think the biggest point right now is everyone gets affected, first of all, on the memory side, especially actually the frontier ones, the ones that are using the CoWos, HBM gets even more. And especially to Sid's point, good luck getting allocation for CoWos and HBM ahead of NVIDIA, then you have TPU Broadcom ecosystem. Then you have AMD, then you have Amazon, then you have Microsoft, then you have Meta. Good luck. So I think it's like ... the point is beyond even the price, can you get access off important supply chain components that are going to make you make compute, which is the actual true ultimate bottleneck because to your point, and kind of just circling back to your original question, your first question around power and how we're going to solve this, like look, from my perspective, I really believe in ... I'm like a really power maximum. We have to deploy as much power and build as much power as we can. It doesn't matter the solutions that Akash does, Positron does, d-Matrix does. It doesn't matter how power efficient we are, how much we save on power, that's all good. That's just adding more to the capacity, but we need still way more power to meet all the demand curves. So obviously, roast tinted glasses here, living in the bubble, whichever way you want to term it, but the point there is just that we need a lot more power if the demand curve is even remotely going to keep on going the direction that it's concurrently trending towards. And so that's that.

Dave Vellante

>> I mean, Sam, there's no indication that the demand is waning. And in fact, of course, data center accelerators are sucking up all the demand because they're making commitments so much larger than say the typical consumer than Apple and the others are not growing as much. So you really have to get into the value chain and figure out where you place your bets. So how do you think about that? And in terms of where do you want to be in that value chain? Do you want to be throughout to hedge your bets? Do you want to sort of double down on any particular zone in that value chain?

Sid Sheth

>> Yeah. I mean, for me, I think I spend a lot of time ... There's two different types of companies. So Felix and his company is a material science company that's servicing, inference servicing training. And I think in that type of market opportunity, there has to be a paradigm shift. So for Felix, diamond is the most thermally conducted material in the world. You're saying, "Hey, it cools the server better than air conditioning. It cools the server better than water could." That's a paradigm shift. No one's quite done that yet at scale, so that's a place I would want to be, or with Positron focusing on ultra specializations for transformers, they're now working with Oracle that became public, not to steal your-

Mitesh Agrawal

>> No, no, you're good.

>> It's public knowledge now. The Oracle CEO mentioned Positron. And so finding specialization, finding where the ball is heading. And if you zoom out and look at the Banyan portfolio, there's strategic bets across this entire new, highly specialized infrastructure stack. So we have Falcomm. I can't say who led their round, but one of the largest space companies in the world, they're the most specialized chip for five and 6G communication or the biological computing company, they just announced their large seed round. They're using human synthetic brain neurons to replace silicon chip development. I was a pre-seed investor and you could just go down the list of the companies I've backed very early on and they all have a fundamental new way of solving a very big problem. And then, if they get that right, it can be a real business, have great margins and add a lot of value to the markets they're solving. So I'm looking for those very early on because those problems have to get solved. We can't just keep doing things the way we're doing them because there's a lot of plateaus and innovation. And I think that's something I see quite a bit is ... especially in infrastructure, like for example, if you go to any of the 5G towers around us right now sending signals to our phone, most likely that technologies are 20 or 30 years old. So who's going to be the startup daring to change that and who's going to back them very early on? That's what I do at Banyan.

Dave Vellante

>> Well, Mitesh mentioned the B word, bubble. Congratulations on the Oracle deal, but you see Blue Owl and Oracle and of course, OpenAI's commitment, but from an investor standpoint, you want to be in the wave. It's worse than not be in than to be in and then at some point, maybe there's going to be some overcapacity, we'll deal with it, but you can't not be in it. Now, I wanted to come back to you, Felix, and ask you a question because something Sam said caught my attention. I mean, you're not essentially obviating the need for liquid cooling, are you or-

Felix Ejeckam

>> No, we're not. No we're not.

Dave Vellante

>> You're not implementing that, right?

Felix Ejeckam

>> Correct.

Dave Vellante

>> Because the curves that we're on are going to require these types of technologies.

Felix Ejeckam

>> We compliment both air and liquid. Now, if you don't have the money to invest in liquid infrastructure, then we can support you and help you get to a 1.1 PUE. But if you do have the installed base infrastructure for liquid, then that's fine. We'll help you get some money back in ROI. TCO will look very good.

Dave Vellante

>> Yeah. If you can get to a 1.1 PUE, I mean, that's good news. I mean, a lot of the enterprises that I talk to are saying, "Well, we have to rethink the data center infrastructure and they're not really set up for liquid cooling. So this could be a bridge for that for sure." And then Sam, you mentioned cell tower. So again, I've started at MWC last week. And John Furrier wrote this piece a while ago talking about the hyperconverged edge, basically compute storage, networking, wifi, fiber, basically a mini AI factory within these cell towers. And it's interesting because, as you know, the telcos have missed every monetization opportunities for quite some time. They're great at connectivity, and of course they wanted to monetize 5G and they kind of really didn't. Are you seeing these types of applications emerge? The one that we saw was they'll help you optimize your phone bill. It's like, okay, what else do you have that AI can help me do? Real applications. I mean, they weren't great at content, but who's going to win that game? I mean, obviously the over the top vendors won sort of the last round. Are you seeing applications emerge at the edge that are interesting? Are you guys just so head down trying to be the enablers and let the rest of the value chain figure it out?

Sid Sheth

>> Yeah. I mean, it has to start somewhere. It always starts in the core, right? I mean, so you put intelligence into the core, applications get developed into the core and then it diffuses from there into many other places. And that is exactly how it's playing out. I mean, you saw a lot of intelligence being deployed in the core. I think there's going to be a lot of intelligence still deployed in data centers and in the core. But I think for the first time, people are now realizing, "Wow, there's an ability to take these models, distill them down, and deploy it in smaller form factor computing applications that are closer to the edge." And what would that look like if all this ... Eventually, the vision is intelligence will really diffuse to pretty much anything that it can diffuse into, right? And it starts in the core, but why stop at anything? Every object that you see around you could have some form of intelligence built into it depending on the environment that it sits in and can make decisions based on that. So in our opinion, look, this is going to diffuse everywhere. As a company, we are focused in the core because that's where the revenues are, that's where the market opportunities in the near term. But the way we've built our platform, at least, it's built with chiplets and it can ... Over time, we can sprinkle these chiplets in a rack and kind of deploy it in a core application or over time, strip it down and just distribute, sprinkle fewer chiplets in an edge application, right? And so, we deliberately built it with that vision that, look, start in the core, let it go diffuse in the edge, and the platform should be built in a way whereby it can be deployed anywhere.

Dave Vellante

>> I want to ask you guys, we have to wrap, but it's so hard to predict, right? Things change so fast, but reasoning kind of saved the business in a big way. If we're not for reasoning, who knows? Maybe the bubble would have popped earlier. We don't know, but what is the near term sort of milestone that you guys are hoping for, optimizing for, expecting? You can't look out three, five years. I mean, you can, but actually maybe it's safer to look out five years than it is 18 months. But are there any particular milestones that you're hope coming out of GTC that you're ... Obviously, Jensen is going to talk about the Groq deal. I'm sure you guys are going to be paying attention to that. Is there one thing that you guys are looking for, expecting, hoping for in the next 12 months? We'll start with you, Sid, and go around.

Sid Sheth

>> I think going back to one of your earlier comments is revenue is being held back because there's not enough energy available, right? So it's what are 10 billion dollars for a gigawatt of capacity, that's the number. And OpenAI has been pretty public about that. "Hey, look, we just don't have enough ... I wish I had more capacity and I could unlock more revenue." So I think clearly the path that we are on is not going to get us there in the sense just brute forcing it, throwing more GPUs. So I think this whole heterogeneous computing movement whereby people actually for the first time are embracing alternate architectures. And I'd like to see some numbers in a year from now that clearly show that by taking that approach, we were able to unlock more revenues for applications that people cared about. We haven't seen that yet, so that's something I'd like to see.

Dave Vellante

>> Thank you. Mitesh.

Mitesh Agrawal

>> Yeah. I mean, for me, the excitement is always on the model development side. I think to your point, reasoning was very important because obviously exploded the amount of tokens, but also it really allowed for problem solving to happen very, very well right now. Continual learning is supposed to be one of the key things for the model, while it trains learns, while it's doing inference learns too. So those are the two spectrums. I think in the middle, I think what we will see in real time is we are seeing context length increase without degradation. I think that's the key for next level of ... for good problem solving, especially around scientific applications around world building. It's like, okay, how do you go from 200K to a million to 10 million context length, but without the degradation that currently happens, even anthropic and open AI that are saying one million context lens, the degradation is very ... Because you have to set up attention over that. So that's kind of really one of the key aspects to me. And I think the second aspect is really around this whole ... whether you call it world models or alternative transformers, whichever way we look at it, I think the world simulation models will be very interesting of not only just having a true physics engine at its core, but then, hey, do they open up new challenges for ecosystems for silicon, right? Do they work great on Positron or d-Matrix or NVIDIA or you need something else for those? And that will be another interesting kind of point to the industry. It's like, "Oh, this is another type of architecture that we need." And I think obviously, we are constantly looking ahead and we try to fit our narrative in there too. But I think as those applications scale up, when you're truly ... rightfully so, NVIDIA and Jensen says that they're going to design these factories in simulation or when he says every pixel will be generated and not render or shot, I think those days when they arrive, I mean, that's ... the level of compute that will require is unfathomable today. And I think how do we address that, I think, and that will be really clear in the next 12 to 15 months, especially around the model growth side of it. So it's super exciting for all of us to actually be a part of it. And this is the key validation to your point, you mentioned about the NVIDIA Groq deal, but not just that, I think over the last six months, as I said, inference is not a secret anymore. And we have to accept and be humble that like, look, NVIDIA is well over 90% of the world's compute supply and they're going to own most of the supply chain but principles for many, many coming years. I think the thing though is that the market is so large that we have to just make sure that our technologies are unique enough and performant enough and that we can source our supply chain for enough of our allocation. When people call me and say, "Hey, can you get enough memory?" Yeah, if I was NVIDIA, I can't get enough memory at that scale, but I'm Positron and I'm finding my way through. Yeah, I can get enough memory for our chip. So I think that's kind of the key point.

Dave Vellante

>> And Sam, you have to have patience. I mean, you know it takes seven to 10 years just to get to a company to a point where it can even achieve escape velocity and that in and of itself is just a real grind. But what are you looking forward to, hoping for in the next 12 months?

Dave Vellante

>> Yeah. So for me, when I take a step back, if we just think about ourselves, we can only learn so much in a day. We can only remember so much. We can only communicate so much. I can only communicate in one language, but if you look at AI, AI can speak multiple languages. AI can work 24/7 ingesting information and then reciting it back to you in a very eloquent way. So I'm really excited about this shift in humanity where for the first time AI is generating more intelligence than humans. And I think if we look throughout history, we've always assembled ourself around infrastructure, whether it was trains or electricity or airlines and like flying. So I think AI and the intelligence it creates is becoming the new norm in our society. It's a generational shift. And so, all these amazing founders that I'm able to work with, just backing them and helping them build out the expertise in their specific areas, I think gets us just a little bit closer each day to that promise of abundant intelligence. And then, that unlocks a bunch of great stuff for society, whether it's in medicine or transportation or helping our environment, all these really great societal impacts come from that abundance of intelligence.

Dave Vellante

>> Thank you for that. And then Felix, it seems like your expertise in material science and your firms, really have stumbled upon probably the single biggest constraint in the business today and the economics from what you said are pretty much durable, but what's your hope for the next 12 months?

Felix Ejeckam

>> So 12 months out, my hope is more of ... Not to be negative, but I'm concerned about some of the existential threats to the revolutions that we all speak of. I think that the war currently going on in the world is potentially problematic. The revolutions that we are all talking about, I think the ball has been set in motion. They're going to happen in the next 12, 24, 36 months. Various verticals are going to see so much tremendous leap in the way that we as humans conduct our lives. Medicine, I think, is going to see tremendous revolutions, but all of that could get tripped if we don't fix this war going on, if it doesn't come to an end. And so that's my biggest concern. This tremendous amount of our supply chain collectively depends on the Strait of Hormuz and that could be affected. We just talked about the price of memory. What if there is no memory to buy for six months? What does that do to all of our manufacturing? So those are the things that concern me about the next 12 months. But I think in the long run, probably things will start themselves out and much of the transformations that we speak of will come to be.

Dave Vellante

>> Yeah. Thank you. We hope and pray for a quick exit to this war that's going on right now. The Middle East looked like it had so much potential and putting in capital. And so we don't need this. So guys, thanks so much.

Felix Ejeckam

>> Thank you.

Dave Vellante

>> Really appreciate your time. And thank you for watching TheCube and NYSE Wired GTC pregame. I'm Dave Vellante for John Furrier and the entire Cube team. We'll be right back after this short break.