theCUBE + NYSE Wired: AI + Cloud Leaders | Mark Castleman, Intel AI + Cloud Leaders

Clips
More from theCUBE + NYSE Wired: AI + Cloud Leaders

Mark Castleman

Managing Director, Intel AI Cloud

Intel Corporation

play_circle_outline AWS summit in New York showcased significant innovations and trends in AI.

play_circle_outline Enterprise AI execution struggles with productivity and real-world application challenges.

play_circle_outline Agents and APIs create opportunities for optimizing enterprise workflows and services.

play_circle_outline Evolution of applications into agent-based models for personalized task handling.

play_circle_outline Potential market shifts driven by younger generations pushing for innovation in Europe.

play_circle_outline Changing landscape of competitive strategies in technology and entrepreneurship.

play_circle_outline Frameworks like Prompt, Context, Model (PCM) to enhance agent workflows.

Info
Transcript

Mark Castleman, Intel AI + Cloud Leaders

Mark Castleman

Managing Director, Intel AI Cloud Intel Corporation

In this exclusive interview from theCUBE + NYSE Wired: AI + Cloud Leaders event, Mark Castleman, Managing Director at Intel, joins theCUBE’s John Furrier at the New York Stock Exchange studio to unpack the enterprise AI landscape at a pivotal moment in 2025. Against the backdrop of AWS Summit New York and leading into re:Invent, Castleman shares deep insights into agentic workflows, AI-driven infrastructure monetization, and the growing complexity of enterprise implementation.

Castleman explores how agent-based architectures and API integration are u... Read more

explore Keep Exploring

What topics were discussed at the AWS summit this year? add

What challenges are being faced in the implementation of AI for enterprise productivity, and how do agents fit into workflow optimization? add

What are the challenges and potential benefits of implementing AI agents in complex workflows? add

What is the concept of using agents in workflows instead of traditional applications? add

What is the current attitude of technical professionals towards the emerging wave of innovation? add

What are the challenges and considerations entrepreneurs face regarding competitive strategies and market advantages in today's rapidly changing environment? add

What is the framework discussed in the talk and how does it relate to workflows and human involvement? add

bolt Powered by CUBE AI

Mark Castleman, Intel AI + Cloud Leaders

search

>> Welcome back, everyone, to theCUBE. We're here at our New York Stock Exchange CUBE studios. Obviously this is our now East Coast hub, we have Palo Alto, and we're connecting Silicon Valley and Wall Street, of course, working with Brian Baumann and the NYSE, creating the New York Stock Exchange Wired community. You see a lot of emails from Brian and the team, of course, building a trust network and covering as much content as we can. This week, AWS was in New York for the summit, and we've been programming around it with our AI cloud leaders. Mark Castleman, managing director at Intel, is here as part of that leader series. Knows a lot about technology and money and culture. Mark, great to have you back on theCUBE. We saw each other in Maui for the Actai pop- up CUBE by CUBE. Now we're in the studio. Thanks for coming in.

Mark Castleman

>> My pleasure. Happy to be here.

>> Yeah, not too shabby. - Oh, it's great.

Mark Castleman

>> Open outcry option trading, a lot of green over there,

>> except for Tesla in the red.

>> I really thank you for coming in, I really appreciate it, because at AWS this year, it's the summit. It's a free event. It's usually their marquee conference for practitioners. It's free and it's really about learning. It's New York, it's everybody. So everyone's in town. This year it felt like a full-blown re:Invent event, which is their flagship primary event. Major news Nova model with customization. They had AgentCore, a full platform, and then obviously a marketplace separate group for listing agents and tools. And they even had good news of this Kiro application, which is a developer environment. Matt Gorman had to release it the day before, so it's like, I won't say throwaway news, but they curtain raised it with that. So I mean, pretty significant news. This would headline any conference and there's a mid-year halftime report, which is part of our show, covering it. So it's our digital twin. This means that the re:Invent, in our opinion, is probably going to be massive. So it just speaks to some of the things we talked about at the RAISE Summit, when we were there, about how much innovation's going on the first half of the year. If this is the early preview ...

Mark Castleman

>> . - Of AWS, what we saw at RAISE

>> and what you've been working on and seeing, it's significant. What is your take? It feels like almost a pinch me moment. I don't believe it's almost this happening.

Mark Castleman

>> Yeah. I mean, look, I think everybody is trying to figure out the enterprise AI execution game and what works. People are on the hunt for what's going to work. There have been a lot of things that folks have tried that didn't quite work. So there've been this delicate imbalance between the pursuit of productivity and AI for enterprise and the actualization of that have not quite aligned. So if you look at what's happening in the agentic space and the tools and the pursuit is the idea that the benefit to enterprises in the workflow, and agents are perfect for workflow. And workflow and workflow optimization and decisioning inside of workflow is exactly what a collection of agents can do very effectively, but it's hard because you have to understand the workflow. So it's not like an AI can just go in and say, "Oh, I've got this solved. " There's going to be, in a simple workflow, you could have 20 agents, 40 agents, 50 agents, and then all of the boundary markers making sure that no one goes wild in any one of those steps. So as a practical matter, it's actually way more complicated than people give a credit for, but those that solve it win massively. It's a big win. It's a very big win. Now the thing that's coupled to agents that I also like to talk about are all the APIs. So from another perspective, once the agents can start wiring up all the available APIs, then you have a monetization path for infrastructure, which is also really interesting because there's a lot of infrastructure sitting out there underutilized. And if the agents can get their act together and we can figure out how to wire them up and consume the APIs, you have a nice business model. Very, very nice business model.

>> What are some of the things that they could wire up? Give an example. Because I think that's a really good point. If you look at, I'm going to throw back my memory, but grid computing was a concept of the '90s that carried forward because it was grid computers talked to each other. What you just basically talked about is grid computing, where the grid of available resource, if connected, can be optimized in a way.

>> Yes. - So explain where you see that happening.

>> Where would it wire up in the infrastructure?

Mark Castleman

>> Well, so as a backdrop for that, what I would say is

Mark Castleman

>> that if you think about the app economy and what happened in the app world is that you had a product manager that was trying to develop the best set of functionality and experiences for the broadest number of people, one to many. And those apps were consuming services, API services, maybe they're trading data or pushing and pulling instructions to a network or getting banking information or getting health information. They're basically dipping into a lot of different data stores to get information or to trigger activity. So that happened in an app. Well, if you dissolve the app and put that capability into an agent and put that agent with a collection of other agents in a workflow, you get app- like functionality out of a sequence of agents. Those agents are also going to consume API. So they're going to turn up and turn down services. They're going to push and pull data, they're going to make calculations, they're going to execute other agents to do other things in some additional workflow. So think of the agent construct as these things are working around as a deconstruction of a single app, but now I can do it for myself. So I don't have to have a single app doing it for 100,000 people. I have one collection or maybe a collection of agents doing something for me. Now the cool thing there is that I have an account at, let's say, my mobile provider, I have an account at my bank, I have health data. So my agent can talk to all of these different things and push and pull data and then set appointments or make trades or do other things based upon other activity. So when I'm talking about the agents will start to activate access to these things, the folks that are running the APIs, whether it's the financial services firms or the healthcare companies or the transportation companies, they now have a completely new revenue model to expose those APIs as access so that the billions or trillions of agents that are out there can start using them. And that is why this is such a massive, massive push because if you can become a greater hub of agent activity, then you can trade into those APIs and maybe there's a toll. Or just -

>> already put a toll up already.

Mark Castleman

>> They put a toll up already. So I

>> think the data guys are in good shape. I think the API guys are in fantastic shape. The constructing the agents is still pursuit, but it's getting better and better and better. And then once you expose that to not just the PMs at a product company, but everyone, then it's going to change how networks work, how service consumption is perceived and actualized. And it's a whole new world. And so I'm not sure anybody exactly knows how it's going to go down, but certainly going to be super interesting.

>> I mean, you and I have talked about this being looking at historical waves of innovation. We've seen many, many waves. This one's huge. The one thing that's always present is the enthusiasm of alpha nerds and geeks. And I got to say, people right now who are technical or have been in the industry all get it. They see it. And they're not just saying, "Oh, this is the next big thing. " They're actually super enthusiastic and excited, smiling, digging in, rolling up their sleeves, getting on board. I'm enjoying it. Literally, surfing the wave and having fun, getting high off it. I mean, it's intoxicating, it's intellectual. Do you agree with that? And what would you observe, if you can look around? I mean certain people, some people are scared they might be on the wrong side of this, but if you're on the right side, like the Amazonian talent, they have a field day here. I was just like, there're senior people, technical people just grinning, loving life right now.

Mark Castleman

>> We talked about this before where if you ever want to identify transformational movements, you have to look at how good they can be and how scary they can be. And the greater the delta there, the greater cultural impact. So if things can be really, really bad and really, really good, you know something is-

>> The symmetry, the symmetrical polarization means it's big.

>> It's big. It's going to affect culture.

Mark Castleman

>> And things that affect culture are, by and large, just orders of magnitude more valuable and more impactful and impact more people. So in this case, whether, let's say you're a more senior entrepreneur or new to the game, it's really exciting because you can turn something very specific into a billion- dollar business and you can very quickly put value into the marketplace and capture that value and do the next thing faster than ever. And there's no longer like this five, seven, 10 year window. It's really more of one, three, five year window. Everything has compressed because you can achieve value much sooner.

>> It's almost scary for the entrepreneur, because almost you have FOMO about not being on time with the wave or do I go too slow? Remember the old days, run fast, mover advantage? Those things are situational. First advantage has certain advantages in markets that have characteristics. What's your take of this? Because I'm a student of competitive strategies. One of my little hobbies is like what's a good strategy, how do I compete and what's a moat? You got all kinds of new changing variables around what used to be yesterday's competitive strategy, open source. And there's no real proprietary moat there. That's been gone for a long time. Nvidia, we always debate about their moat. Is it a moat or is it just scale? What is it moat? I mean-

>> Big moat.

>> So what's your take on this?

Mark Castleman

>> Because I think there's a new set of forces. Even Amazon, I scratch my head and go, "What's in Andy Jassy's head," because he just did a video on, he's not even charging. He still loves AWS. He's done a video on AgentCore and really describing the benefits. He's excited. This affects Amazon. What's their strategy? Just continue to sell more? So this could hurt them, it could cannibalize them or they could win big. Maybe they're from your camp, win big.

>> Yeah, it's a big, these are defining moments

Mark Castleman

>> because it's part of that race where the winners are more energized and the ones that aren't going to win are going to be confused. And so clarity is super important. Quickly making decisions, being clear about what you're trying to achieve and then executing towards that is super important. Any pause is going to be a major problem. It used to be first mover advantage, I'm first to market or whatever. Now it's the first news of the day. So you literally have to be the early bird that gets the worm at the beginning of the day, because by the end of the day, something could have changed that could affect your entire business. And what was it, the company, Windsurf, that just went through this three-day acquisition, cut a deal, do a partnership in three-day cycle and multiple billions of dollars involved. And that kind of velocity is just unbelievable to imagine at that scale. But that's what it's going to take to gain ground. But it is exciting.

>> What do you think about a couple things.

>> Obviously you saw the Meta, big movement around offering people $100 million. There's a lot of flack that basically the investor's has got nothing from the deal on Scale AI and other deals. And then you got TSMC's earnings are out. Up 61% on net income. They've been beating estimates every quarter since 2021. They're upping their construction on Arizona, the Silicon Desert play, where Intel has a factory, you know that. And then, so these are some of the headlines and ChatGPT debuts their agent, which could take over your computer. So you just gave a talk at MIT on agents. You see the expansion of the infrastructure in the US from TSMC. You see the hiring piece. It speaks to the culture we're living in. I mean information claims that they call it free agents, but we were the first ones to say tech athletes and that there were free agents, but it's happening. The tech is happening and there are now tech athletes on a four-year .

>> Yeah, that's right. That's right.

Mark Castleman

>> $500 million. - $100 million contract.

>> It's a four-year cliff, go.

Mark Castleman

>> And it won't be long before Congress gets involved

Mark Castleman

>> and starts collective bargaining

>> and all this stuff on the athletic tech, the tech athlete. That's a very interesting concept. Actually, that's a business there, by the way.

>> Yeah, definitely. - We should become agents for

Mark Castleman

>> that business, that we could represent-

>> If you're a PhD student, contact us.

>> . - You'll be the Scott Boris-

>> NIL for tech bros

Mark Castleman

>> and gals that want to go do the $100 million deals

Mark Castleman

>> with Zuckerberg.

>> Well, look, if you look at ESPN's history, they

>> really broke out when free agency hit sports because they were an up-and-coming broadcaster and the agents can earn a lot.

>> . I'm telling you-

Mark Castleman

>> Leaking a lot of information to the ...

Mark Castleman

>> That's right. There's some there for sure.

>> Well, if you think about just the overall context of the ... land grab is an overused term, but obviously the capture value side is so critical. You have to aggregate the best possible resources, and that's why it's acquiring, whether it's compute, power ...

>> Data. - Data and humans.

>> Yeah, I think that's a moat question right there.

Mark Castleman

>> Power could be a moat. Data's a moat.

Mark Castleman

>> Humans and team

>> and intellectual strength has always been a moat. And it's harder and harder, I believe, to create a moat in that space, as the tools become easier and easier to use. And if you're leveraging AI, this notion of a 10x engineer is obliterated. So I don't know-

>> In what way?

>> Because that was a cloud construct.

Mark Castleman

>> Well, because the idea was that all of the capability was encapsulated into a single human who was just unique in their ability to execute in ways that other people couldn't. But a lot of that 10x capability, which is access and experience, is encapsulated now in AI models. And so what does that mean in terms of what these individual people are capable of doing for any specific business? And I think that's a degrading piece of material over time because the other capabilities just catch up so fast. But I get why they do it. I totally get why they do it, and it's hard to argue with it when you realize that the prize is really, it's a huge prize. So imagine two years out, and Facebook, sadly, to use that example, I'm not trying to clear where it's going, but if it's a big win that looks like a super insanely smart spend. So we'll see.

>> I mean, if everything's open and free, then a moat like having the most GPU intelligence, horsepower could be an advantage. If you have data moat and you then can configure it properly, then that can be a moat. Now, I've got a whole nother perspective on this because if you asked me a year ago, or even a half a year ago, I would've said, like our business, theCUBE, I need to have a GPU farm and do a deal with Lambda. I'm going to go crack and get arms rigged. I need arms, I need nukes.

Mark Castleman

>> Yeah. - Yeah. If you look at what Amazon just announced

>> and what we're hearing in the AI circle, I'm curious to get your thoughts on this, is that the LLM might be passe, in the sense of it being the monster of value. And let me give you an example. One of the big trends this year that's emerged is MCP. Now you got A2A, agent to agent, for multiple agents swarming. The other big surprise is tooling is growing huge. In fact, Amazon has an agent and tooling section of marketplace. It's also popular. And what they're doing with tooling right now is using a FinOps model to say, "Hey, let's maximize our token. Why ask basic math to LLM and burn tokens? Let's use the tokens for abstract reasoning only and then create software layers with tooling and agents to plumb out non-related requests that tokens don't need to be used for. And I'll use compute for that. I got plenty of compute. " I mean, Intel's in the compute business. So this could be a dynamic that could change conventional wisdom of overbuild with the GPUs.

Mark Castleman

>> Yes. Okay. So ...

>> What's your thoughts on that? Am I crazy?

Mark Castleman

>> No, you're spot on. Okay. So it starts with this.

>> It used to be like throw everything at the LLM and just deal with the cost, because it's so good. But now, to your point, if you take any prompt, you do a reasoning analysis of the prompt to determine which model you need to use based upon the price of the model. So some prompts don't require the big model, and they don't require deep research or they don't require these other things. So you can get a really good output for very low cost very quickly. But now if you turn around and you take a much more complicated prompt, a reasoning analysis on that prompt says, "Hey, this is a very difficult problem. " It doesn't answer the problem, it just understands the complexity and says, "I'm going to give this to a much more capable model. " Token cost goes up, time goes up, but the probability you get a good answer also goes up. So that's like a traffic cop, to figure out where you put this stuff and the whole idea there is to lower cost and increase the probability you get a good answer for the lowest price. Now, the problem in that is that not all tokens are equal. So this whole idea that token cost is coming down is true. But what that demonstrates is that the tokens that come out for one prompt have a different cost per token per watt than the tokens that come out for another prompt. So how do you know in advance what the cost of any particular response is? And this is the problem for enterprise, because if you have an outcome you're trying to achieve and you're trying to build an implementation that has a benefit that you can quantify, you also have to know what the cost is. Well, if you don't know the cost, you can't uncalculate ROI. So you have unlimited risk, basically. So this is what's killing enterprise AI. That's why POCs are dying, the implementation of AI and-

>> Interactive trading behind us .

Mark Castleman

>> Creative trading. - That's it. Nvidia is up again.

Mark Castleman

>> I can't help but think about trading places.

>> Get those brokers back in here.

>> Turn those machines back on.

Mark Castleman

>> Exactly. So I mean, look, it's a long-winded way of saying

>> that until you and can simulate in advance and be aware of the cost per token for any particular outcome you're trying to achieve, you don't really know if you're getting a benefit.

>> Basically my point, and you agree.

>> So what you're bringing up too is that we still don't know what the ideal configuration will be, or it could be very ad hoc runtime or it could be agent-driven based upon some programmability.

Mark Castleman

>> Yeah, enterprise PSCs go like this. Hey, I would like to do X, Y, Z, fraud detection, churn reduction, defect detection, whatever. Okay, so let's go stand up a bunch of compute at some amount and then throw that at the problem and see what happens. And I will tell you there's a lot of disappointment in that approach because what happens is very different, where the cost of how to get to what happens is not what is expected. So then the projects get killed. So I think, as we drive out the ambiguity of the cost per token per watt, you get closer to what compute do I need? What model do I need? Where does it need to run? How close does it need to be to the things that I need to access for time and return?

>> At the end of the day, the KPI is task completion.

Mark Castleman

>> Task completion at a price over time that has a probability near perfect. And some outputs require 100%. In a probabilistic model, maybe the model isn't even what you need. You need some deterministic approach. So it's not clear that the LLM solves all the problem, which I think is the point. However, what's really cool is that there are good enough models that give you very good answers that can run very tiny in different places, consuming less compute. And if you bound it with the right prompts and the right guardrails, you can get really good answers out of that, that fit your workflow. So this leads to a distribution effect, so you can distribute your compute requirements out onto the network. And that was why I was at MIT, because Ramesh Raskar, who has this lab there, is working on this thing called NANDA, which is this distributed approach.

>> Explain what . Explain what was the event,

>> what was your talk, who was there, what was the vibe? Yeah, you're like a reporter. Good, go. Give us the data.

>> Yeah, on the ground, I'll tell you how it was.

Mark Castleman

>> It was a cold day that day. No, it was a super hot in Boston, but it's essentially the idea that eventually, disaggregation occurs. Eventually, things spread out. They start out concentrated and then they spread out. And now that's like a physics model, but this is true with tech, with, let's say, cultural impact, financials, et cetera. Things tend to spread out. So the theory is that eventually the compute will spread out, the models will spread out, the capability will go out to where it needs to go, and this concentration is artificial. So my talk was not necessarily about that specific thing, but in that possible reality, what can you do now that can work in both? And so you mentioned MCP. MCP is a protocol, but my talk was more like a framework. So if you, and I called it-

>> Framework for agents?

Mark Castleman

>> For agents. So I called it a PCM. So it was like a cute little reversal. So prompt, context and model. So this is a framework, not a protocol. The key in a workflow is a prompt. The prompt is the human understanding of the workflow. You still need a human to write the prompt, and your prompt is actually your greatest guardrail for any output that you get from a model. Now in a workflow, we built some of this for Intel, is that if you take a single use case, we have this query engine where the agent, the agentic tool, steps you through the workflow and at each point, collects more that it knows about that particular spot in the workflow and asks the human, "Are we getting closer and closer and closer to this finish? " So we can take a workflow that takes four to six weeks, and we can do it in 10 minutes because we have 100 prompts that know the questions to ask at each stage. We got a constellation of models that are checking the data that comes out. We've loaded all of our historicals on this particular problem, so we know what the metrics are that are good and the metrics that are not so good. And as we tune this thing, at the end of it, in 10 minutes, I can get a full-blown CSP class.

>> So PCM is the acronym? - Prompt, context, model.

Mark Castleman

>> Model. - Prompt is the human understanding

Mark Castleman

>> of the workflow, context is understanding the impact

>> and then model is model.

>> So in your use case, by the way, first of all, that's a

>> great illustration, but I have to just drill down on that. The assumption is that you have to have the domain knowledge of the workflow. The prompts prompt the person who actually is the domain user?

Mark Castleman

>> Yes. And that's how you build the agent.

>> Who sets that up? - Well,

Mark Castleman

>> think about the agent is a proxy of the best person or collection of people in the world that can do that thing. Well, now we have to go get the people. And you can synthesize this a little bit, but you still have to have a person looking at that going, "Yeah, this is how it works. " It's almost like labeling on data, but you have to get the prompt correct. A human and an AI can build prompts. So there's prompt engineering and prompt that the synthesization. Synthesization is not a word, but you know what I'm saying.

>> Yeah. - And then what you can do is you can build this

Mark Castleman

>> prompt, and if you're a human and you read the prompt, it should read the most expert person trying to solve a problem. And then the model has access to the data and the model can find the relationships between those points-

>> What's interesting around MCP is that it allows you to have that protocol to do that. Now, what's come up with MCP is, because it's so popular, they're everywhere. And so they're a node on the network and they're important, trusted nodes. So what came out of Amazon was, and some of the other people I talked to was, people were building their own MCP servers, and now Amazon has actually built a better one. An Amazon person told me, that was doing the MCP servers, and say, "We just want to build the best one because we don't want people reinventing themselves. " So what's happening is everyone's creating their own MCP servers from scratch.

Mark Castleman

>> Yeah. And that's a huge problem too, because security is a, I've seen issues where I'm using MCP to solve a problem, and the model's like, "Hey, that's a great answer for this other person. " And it starts to blend output, and that is bad. So now you can't have multi- tenant solutions and all this stuff. So everybody trying to do their own thing, roll their own, whatever you want to call it, can be a problem. Yes.

>> Mark, great to have you on. Just one final question. I know you always have a money side, the finance. You have a background in looking at the capital market's impact. Just as the speed happens, is there any black swan event coming in money side of it, hedge fund, private equity, big country invests, sovereign investors? I mean, you get the Middle East, you get China and the US, I see as the three pillars of where AI's going to land. I would argue that the Middle East, they love their sovereignty and just want to sell oil to people and AI's the new oil, as they say. But it's China and the US. You got those three access. Then coming out of Paris, don't count the Europeans out. There's a revolution going on over there.

Mark Castleman

>> For sure. - The younger generation's saying, "Hey,

>> all you compliance people, all these rules," that's my vibe, by the way. No one actually, there's no evidence other than me reporting that, but that's the vibe. Europe's like, "We're going to be left in the cold. " But right now it's the three.

Mark Castleman

>> Yeah, look, I mean, obviously the US is in a very strategic position, but you're right. The Middle East has a lot of capital and a lot of desire, not only to play in that world, but to impact it and to build their own ecosystem. So not only they want an ecosystem locally, but they have export goals, and there's a big vision over there. China is clearly, they're pressing hard. Pressing hard on power, compute. They want to be competitive and they're going to be a competitor. And the US has a great foundation built on it, and this is going to be the case for the next generation. But I wouldn't count out ... like you said, Europe is really pressing hard to be in that game. I think they're in that game in a lot of different ways. There's a lot of great companies coming out. I've seen great companies coming out of France, Spain, and the UK, and not to leave the others out, but so many great entrepreneurial events coming out of there. As an investor, I look at that as an investible space. Entrepreneurs see capital and they see customers, so they're going to move around wherever the capital and the customers are. So where they end up is different than where they start, often. But I think that, look, you're going to find innovation developed out of need, and there's a lot of innovation going on there.

>> Yeah. Mark, thanks for coming on. You're a leader, manage director at Intel, but also you're out in the industry. Congratulations on that talk at MIT. I want to hear more about that later. Thanks for coming on and being part of our AI cloud program.

Mark Castleman

>> Yeah, you bet. Let's .

>> Okay. This has been the Media Week here at theCUBE, NYC Wired around AWS's Summit event. But all week, we've been talking about AI and cloud, of course, generative AI has its line of sight on the economic value and continuing evolution of the infrastructure and business transformation. I'm John Furrier, host of theCUBE. Thanks for watching.