Brandon Hootman of Caterpillar and Richard Ahlfeld of CoreWeave join hosts John Furrier of theCUBE Research and Dave Vellante of theCUBE Research at CoreWeave Fully Connected 2026 for a keynote and interview on physical artificial intelligence, abbreviated AI. The discussion covers construction autonomy, multimodal models, simulation-driven training, telemetry and fleet data, agentic workloads and the evolving AI infrastructure required for unstructured job sites.
Hootman describes Caterpillar's Cat AI assistant and emphasizes data labeling, synchronized telemetry and on-device inference to reduce latency and improve safety. They discuss the shift from deterministic to probabilistic systems in unstructured environments and the role of simulation-driven training and multimodal perception for construction and mining operations.
Ahlfeld identifies the need for specialized infrastructure, including graphics processing units abbreviated GPUs and central processing units abbreviated CPUs and storage and simulation stacks, to support physical AI workloads. They highlight partnerships among CoreWeave NVIDIA and Caterpillar, the impact of agentic workloads and AI factories on equipment operations and digital ecosystems, and the importance of data sovereignty and a continuous learning loop that shortens annotation-to-deployment cycles from months to hours.
This keynote provides practical insights for infrastructure architects, robotics engineers and data scientists working on autonomy for construction and mining operations.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
CoreWeave Fully Connected 2026. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Register for Coreweave Fully Connected 2026
Please fill out the information below. You will receive an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for Coreweave Fully Connected 2026.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
CoreWeave Fully Connected 2026. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Sign in to gain access to CoreWeave Fully Connected 2026
Please sign in with LinkedIn to continue to CoreWeave Fully Connected 2026. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Brandon Hootman, Caterpillar Inc. & Richard Ahlfeld, CoreWeave
Brandon Hootman of Caterpillar and Richard Ahlfeld of CoreWeave join hosts John Furrier of theCUBE Research and Dave Vellante of theCUBE Research at CoreWeave Fully Connected 2026 for a keynote and interview on physical artificial intelligence, abbreviated AI. The discussion covers construction autonomy, multimodal models, simulation-driven training, telemetry and fleet data, agentic workloads and the evolving AI infrastructure required for unstructured job sites.
Hootman describes Caterpillar's Cat AI assistant and emphasizes data labeling, synchronized telemetry and on-device inference to reduce latency and improve safety. They discuss the shift from deterministic to probabilistic systems in unstructured environments and the role of simulation-driven training and multimodal perception for construction and mining operations.
Ahlfeld identifies the need for specialized infrastructure, including graphics processing units abbreviated GPUs and central processing units abbreviated CPUs and storage and simulation stacks, to support physical AI workloads. They highlight partnerships among CoreWeave NVIDIA and Caterpillar, the impact of agentic workloads and AI factories on equipment operations and digital ecosystems, and the importance of data sovereignty and a continuous learning loop that shortens annotation-to-deployment cycles from months to hours.
This keynote provides practical insights for infrastructure architects, robotics engineers and data scientists working on autonomy for construction and mining operations.
Brandon Hootman, Caterpillar Inc. & Richard Ahlfeld, CoreWeave
Brandon Hootman
VP of Physical AICaterpillar Inc.
Richard Ahlfeld
SVP, Physical AICoreWeave
In this interview from Fully Connected 2026, Brandon Hootman, vice president of physical AI platforms and construction autonomy at Caterpillar, joins Richard Ahlfeld, senior vice president of physical AI at CoreWeave, to talk with theCUBE's John Furrier and Dave Vellante about the infrastructure powering physical AI and construction autonomy. Hootman explains how Caterpillar's autonomy systems evolved from the structured, deterministic world of mining to the chaotic, unstructured nature of construction job sites. He traces the company's path from its Cat AI A...Read more
exploreKeep Exploring
How has Caterpillar's experience with autonomous systems developed, and why is applying those systems in construction more challenging than in mining?add
How is AI being applied to the company's equipment and job sites, and what does the Cat AI Assistant do?add
How does building "Physical AI" systems (for example, autonomous excavators or mining trucks) change the required data‑center and compute infrastructure compared with training foundational models, and do you use a simulation stack (e.g., NVIDIA Omniverse) for development?add
How do you integrate AI into products and operations so that massive telemetry data (from GPUs and physical sensors) can be labeled/annotated and used to automatically detect failures and drive product evolution?add
When did you begin working with CoreWeave, and what motivated that decision—was it primarily due to GPU availability and quality?add
Brandon Hootman, Caterpillar Inc. & Richard Ahlfeld, CoreWeave
search
John Furrier
>> (INTRO)>> Well, I'm John Furrier here with Dave Vellante here in theCUBE's livestream in San Francisco for CoreWeave Fully Connected, one of two days of wall-to-wall coverage. Their first inaugural event bringing customers, leaders together. This next session's about physical AI. I'm super psyched. I have one of the keynote presenters on stage talking about it. Brandon Hootman, VP of Physical AI Platforms and Construction Autonomy at Caterpillar, NYSE listed company. Thanks for coming on. Really appreciate it.
Brandon Hootman
>> Pleasure.
Brandon Hootman
>> Great to be with you guys.
John Furrier
>> Richard Ahlfeld, SVP Physical AI at CoreWeave. Okay, injecting intelligence into something has been the biggest theme. Get beyond the chatbots, but the real meat on the bone is, okay, I'm going to inject intelligence into operations, equipment, data, edge of the network. These AI factories are a complete system of intelligence. So when applied, you get outcomes. So tell us about what you guys are doing. You were on stage and you were talking about some cool stuff. Hey, we're doing our own models. We have a lot of data. Take us through your worldview and what you're working on right now.
Brandon Hootman
>> Yeah, sure. Maybe a little bit about how we got to where we're at today. And Caterpillar's been doing autonomy for a while, and I had shared a little bit about how those systems worked in the past. They were very deterministic. We had them operating in environments that were relatively structured. So the way you built those systems is a little different than some of the other markets that we serve. So if you look at a mine site as an example, Once that mine site gets instantiated, it does change, but it doesn't change frequently. So you have the opportunity to have a more structured, hierarchical autonomous system work and assist on that job site. You go to construction as an industry. Yeah, polar opposite of mining. Unstructured. Could be doing, you know, construction comprises residential, commercial infrastructure development, and highway development. So if you think about all of the dynamics of what has to happen, taking a structured system and matching it to an unstructured environment, it's really, really challenging to do.
John Furrier
>> I was talking with a company yesterday in Palo Alto at Bank of America's investor conference, a hot startup. They got a Series A, so they weren't really in production yet. And the conversation was like— and I said, oh, construction is really deterministic. Great for agents on paper. He goes, it is the most unstructured environment because he was pointing out it's very chaotic. Things are happening. It's very dynamic, but it's kind of a controlled environment. It's not like a Waymo where there's a lot of chaos. And he says it's actually a perfect use case for intelligence.
Brandon Hootman
>> It is. It is. And the, one of the, and we had talked a little bit about this before we kicked off, one of the areas that we started before we went immediately to physical AI is our— I was in our Cat Digital organization and we built a digital ecosystem. We identified that ecosystem and then we built what is the Cat AI Assistant. That Cat AI Assistant lives anywhere our customers live, including in the cab of the machine. So that was really our first instantiation of what I would consider physical AI. And that AI assistant understands the operations, the job site, and it helps the operator be more effective in doing their work. Well, that then really parlayed into— oh, this technology is moving really quick— the probabilistic nature, the reasoning nature, the ability now to be multimodal and observe the environment around and then take action in a particular piece of machinery or have machines working together. We're not too far off from that.
John Furrier
>> Scope, just scope for the audience, just in the past 3, 4 years, say 3 years, the difference of what's changed the most because you just basically talked about the environment around you. So I'm assuming computer vision.
Richard Ahlfeld
>> Yeah.
John Furrier
>> You got models, you have a ton of data. Yeah. What's been the biggest change? Where were you 3 years ago on the progress bar?
Brandon Hootman
>> Yeah, I think that's a great question. we were, we've really leaned into technology as a function of our digital and tech, especially the customer and dealer-facing side of our business. And when I would say that ChatGPT moment hit, we knew that there was a spark and there was something there. So I had the opportunity to start to lead and to develop our AI organization. And really the first use case that we took, you look at it now, it kind of feels like you're rubbing rocks and sticks together to make fire. But our first
John Furrier
>> was—>> Discover the wheel.
Brandon Hootman
>> Yeah, we've got our machines, when you see our machines, they are engineering marvels. They're really large and they can be complex. So when you have a potential failure event on them, knowing the exact service action to be able to take to get that machine back to operation as quickly as possible was a real use case for us, our customers, and our dealers. So that was our first application of generative AI. Just go look at all of the prior service history based on what the potential failure is and make a recommendation on the optimal way to go fix it and the parts required to do that. Immense amount of value to go do that.
John Furrier
>> If you don't mind, David, one quick follow-up on that, because I think this is— I want to get this in. What was the cultural impact of the organization? Because I'm finding in all my interviews on theCUBE the past year and a half, when you have these moments, the motivation on the team is one no one talks about, but it's like the consistent pattern is the spark, the moment, it's just contagious. Explain what happened in the culture on the team. What did it energize? What did people do? What was the reaction?
Brandon Hootman
>> I think it was probably everything that you just said. Also with a little bit of, what is this? What does this really mean to the future of how we build digital capabilities and our customers interact with them? Because if you think about everything leading up to that point, what digital meant was obviously you build data capabilities at scale and then you build unified applications on top of them. So the window into capabilities was screens, menus, clicking. Now all of a sudden you can unlock those same capabilities without having to have someone go open an application or click deep into a menu. So really one of the— and a credit to my peer in our digital organization— one of the first things we had to do with the team was don't fall in love with your application, fall in love with the problem that we're helping our customers solve. And there's going to be a— it's going to continue to evolve how that problem gets solved and how
John Furrier
>> technology—>> clients are ready to say, thank you, Richard.
Dave Vellante
>> In the keynote today, Brandon laid out these four macro trends. And I want to sort of connect to sort of your role. Growth driven in large part by infrastructure and data centers. I told John in our kickoff that Caterpillar was the first stock I ever bought and never sold it. So well done, you guys. Keep it up. And the second was declining productivity despite that growth. Jobsite safety. Construction employs under 5% of the workforce but accounts for 20% of worksite fatalities and then skilled labor shortages, particularly for machine operators. So Brandon, you kind of set up that technology is in part sort of the answer to how you address these. So Richard, where does your role fit and how does technology fit in terms of addressing these trends?
Richard Ahlfeld
>> So from my side, we're both called VP Physical AI. I address the side from a CoreWeave perspective. And if you look at these trends and you're trying to build an autonomous excavator, an autonomous mining truck, then the underlying data center infrastructure that you need to actually run this changes fundamentally. And neoclouds have so far been built first to train foundational models, thousands of GPUs to train GPT-4. Then we sort of went into the next phase where we're trying to optimize for agentic workloads where you have to do inference at scale while agents run through nights and weekends just computing and computing. Physical AI now is an entirely different beast. For physical AI, right, if we think about what they do at Caterpillar, right, we talked about telemetry data, we talked about camera data, vision data. That requires, first of all, a lot of storage. It requires a different infrastructure. It requires CPUs for preprocessing, then If we don't have the right data in the field, we'll have to run simulations. How do you run these simulations? A different GPU, a different compute type that actually runs all of these simulations so I can run virtually, right, a million times a specific digging scenario. And then on top of that simulation run, I need to do reinforcement learning. I need to actually check that when I try to pick up this big rock, it worked or it didn't work. And so you have a reinforcement learning loop on top. And so The infrastructure for the AI learning loop, if you look at physical AI, is a lot more complicated and there's a lot more different steps to it than if you just look at foundational model training.
Dave Vellante
>> So are you using a simulation stack? I think about, NVIDIA's Omniverse or Isaac?
Brandon Hootman
>> It's a great question. And, we had announced a pretty strong collaboration with NVIDIA and we reinforced that at CES this year.
Dave Vellante
>> Oh, okay.
Brandon Hootman
>> Their— they call it their physical AI factory is really the foundation and the core of how we're doing what we do at scale. And then you take the combination of that platform plus the expertise that CoreWeave has in being able to scale, operate, and apply that. That's where this collaboration between NVIDIA, Caterpillar, CoreWeave really starts to come together to help us do some of these things that really haven't been done in the industry before.
John Furrier
>> Yeah, you guys are at the frontier and I just love the physical AI story. I just see it exploding because it really is about the edge too. And, AI factories have basically been dominating the conversation around the large industrial centers producing intelligence. Now, in a distributed computing paradigm, which is essentially hybrid cloud. Yeah, you essentially have distributed computing, you have nodes.
Brandon Hootman
>> Well, and if you think about where our machines operate too, they operate in some of the harshest conditions around the world. A lot of times infrastructure isn't there. So as you think about how these machines are working, you can't count on having network connectivity back. So that edge compute, that asynchronous nature of how the platform will work is absolutely critical.
John Furrier
>> And there's a trend towards more efficiency on power on this, on some of these new chips coming out at the edge and robotics, whether it's humanoid people love or just basically physical devices, trucks. Yeah, large edge. I mean, so we're super—
Brandon Hootman
>> I would add cooling too. If you think about cooling in the environments that we operate in too, that can be an interesting challenge.
Dave Vellante
>> You were talking earlier at the top about the determinism of mining and the non-deterministic nature of construction. We talk about it all the time in theCUBE, bringing those two worlds together. You talk— we talk about application software companies trying to bring those worlds together. Do you think about them as sort of separate initiatives? Are there examples where you have to bring the deterministic and the non-deterministic together? And if so, do you have to build the abstraction layer or are you looking for your vendors to do that?
Brandon Hootman
>> So it's a really good question. We view the architecture between those two not as being diametrically opposed, but the combination of those two is where value can really get unlocked. If you think about things like functional safety, if you think about things that we've developed already that run on the machine like payload systems, grade systems, those are all things that are relatively deterministic. But then if you start to put things like a VLA in conjunction with those where it can perceive the world around it, then we can take those things and then still have our functional safety system working, providing guardrails to the models as they're working. So just the nature of throwing a probabilistic model on a job site and letting it go like that, that would also be a little bit of a recipe for disaster too. So the combination of these two things working together in conjunction, we believe there's a lot of value.
Dave Vellante
>> And you're building that IP, is that right? That's your alpha right there. Amazing. I have a, if I may, we were talking offline, it's really not related to AI in particular, but it speaks to how your ecosystem and your customers work with Caterpillar. We went into our, we have a data partner named Qualitate and I went in and said, what can you tell me about Caterpillar? Is there any information in there? And this customer who's a large construction company in the US said, if I want to consume a dataset from Caterpillar, about oil temperature of my dump trucks. I've got thousands of dump trucks out there on construction jobs. A lot easier to federate that data and share it between tenants than it would be with a legacy approach where I have to build an interface between companies, run it over ETL tools, and absorb data from Caterpillar every night and then pull it into our database. He's talking to— this individual is talking about a sort of modern way to do it. They're a Snowflake customer as well. That is a whole new world, isn't it?
Brandon Hootman
>> It really is. High speed, sharing data. We've always prided ourselves on being customer-focused and just passionate about helping them solve some of their toughest challenges. What's been really great over the last few years is we've taken that from a physical world of iron and servicing iron to now how can we digitally be easier to do business with? And that means we're building platforms. That means those platforms are open, they adhere to industry standards. And when customers need those capabilities outside of our ecosystem, being really, really easy to integrate with and do business with.
Dave Vellante
>> So much data in your systems, the telemetry of that, pumping it into my ERP system and harmonizing all that.
Brandon Hootman
>> Yeah, our digital ecosystem. So if you think about the digital ecosystem and this is before we do anything with physical AI or autonomy, That ecosystem today is roughly 18 petabytes of federated data. That includes data coming from machines operating in the field, that comes from our dealer network, and that comes from customers that actively participate.
Dave Vellante
>> And I presume you're not uploading that into open LLMs, or—
Brandon Hootman
>> We take data sovereignty and protection very, very seriously on our behalf and our customers' behalf.
Dave Vellante
>> And again, that's your alpha and your customers' alpha.
Brandon Hootman
>> That's right.
John Furrier
>> you guys are taking, accumulated industrial expertise and turning it to scalable intelligence. That's
Brandon Hootman
>> basically—
Brandon Hootman
>> We feel that's our value add.
John Furrier
>> That's a major moat. Okay. So if that advantage is centuries of, a century of operational data and human expertise, the question is, how are you thinking about encoding that into machines? What's the vision? Because this is where it comes together. This is, I think, going to be the big 2027 story. I got all the expertise. I got my sovereign approach. It's my data. It's my scalable intelligence, my alpha, my moat. Now I got machines. You got trucks and equipment. How do you encode that?
Brandon Hootman
>> Richard highlighted it. And boy, I wish that there was an easy path to solve the first mile of what we have to do. That 18 petabytes that we've collected so far is just a drop in the bucket to the amount of data that it actually takes to go train physical AI to operate in a world like a construction site. So now you're looking at— you've got to collect not just perception data. Now you've got to collect that data and it's got to be synchronized with your control system, your performance system, your subsystems operating on the machine. And you're bringing that back at a rate of multiple times per second. So if you imagine multiple cameras now, because the reality of what we'll build in the future is it will be vision-based, probably reinforced with things like LiDAR. But you take LiDAR data, camera data, the machine control data and performance data coming back in, you're talking about terabytes of data within a given day for just one machine.
Dave Vellante
>> And can that, um, 18 petabytes, can it help you produce synthetic data that can inform you for new use cases?
Brandon Hootman
>> Helps with some things, but I would also add that what it takes to really train autonomy and robotics really does require that perception data of what's happening at a job site. Yeah, on the machine, around the machine. And then what's really important is what's happening in the machine. What's the operator doing? How's the machine responding to that? How much load was put on the machine? And when the bucket was half full, with certain types of materials. So all of those things, you want to get as much of that as you can, and you want to get as wide a variety as you can of that. And we also think too, that's where our active fleet and our close collaboration with our customers gives us the opportunity to do that.
John Furrier
>> The big story is you're basically operating intelligence in the real world.
Brandon Hootman
>> Yeah.
John Furrier
>> Richard, ball's in your court after what he just said. Okay, now you have to step up to the plate, okay, and take care of this because what he's got is a huge challenge. Latency, safety, uncertainty, tons of data coming in, learning system. This is the learning loop we're kind of hearing on the keynote. What's your direction? Because this is converging down the path where he's going to be ready. Intelligence could help him get there. What does CoreWeave as the partner
Richard Ahlfeld
>> do?
Richard Ahlfeld
>> in particular in my team, we've been building agents that can take telemetry data across petabytes. In our case, it obviously comes from GPUs, right? We have cooling sensors, we have information about electricity and current, and it can look at this colossal telemetry data and it can go through our hundreds of applications engineers' knowledge of why something failed on a particular day at a particular time for a particular reason and connect these two things so it can identify a failure automatically and the agent can go into our CTO Slack channel and be like, last week he said that's something that needed to be fixed and bring it all back together. And so when you said, how do you bake this in? At CoreWeave, if I take a data center at the end, it's like I'm a physical AI VP because I also build physical AI internally. That is a technology we've already built that directly applies to Caterpillar's telemetry data as well.
John Furrier
>> And so connecting this domain expertise with the hardware knowledge, it's just scratching the surface. I can imagine your own product will change.
Brandon Hootman
>> Yeah, yeah, it will.
Richard Ahlfeld
>> It will.
Brandon Hootman
>> It'll absolutely evolve as a result of, this technology getting introduced. But just maybe to build on what Richard said and just think about this problem of what I highlighted, which is the amount of real data we've got to go collect from the physical world. Well, this is data that really no one understands. So the first thing you've got to do is you have to label that data and you have to annotate that data because inherently a model doesn't know what to do with it. In conjunction with CoreWeave and NVIDIA, we've got AI models actually doing annotation and labeling. I don't have to have experts sitting there doing that. So the expert's job is to get the model good enough that as that data is coming in, you can run a pipeline on it to annotate it and label it to actually describe what's happening in the world around the machine.
John Furrier
>> And just what does that change in terms of capabilities and what— how does that move the needle? I know it does move the needle. What's the scope of that? What was the alternative?
Brandon Hootman
>> Well, the alternative was you hire a bunch of people to go do that, right? And then you live through that time that it takes you to go do it, and then you run it through the loop and you find out some things work. Now you're taking things that were months to maybe weeks.
John Furrier
>> Months. Yeah, weeks.
Brandon Hootman
>> And now you're getting it down to hours and your feedback loop between that happening and being able to use it in your simulation environment your training environment happens within the given workday.
John Furrier
>> That's why I like this living learning system that's being talked about.
Brandon Hootman
>> Go ahead.
Dave Vellante
>> Some quick rapid-fire questions. When did you start working with CoreWeave and what brought you there initially? Was it GPU availability and quality?
Brandon Hootman
>> Just this year.
Dave Vellante
>> Yep.
Brandon Hootman
>> So the effort that we're working on with physical AI and construction-based autonomy to start with here, we knew that it was going to be a really tall undertaking for us. And we knew the volume of data that we were dealing with, the volume of GPUs that it would require to train that would be significant. That was one aspect of it. The other aspect of it was guys like Richard that are sitting next to me here, guys that actually understand the application of this, because keep in mind, we've got really solid technical chops within Caterpillar, but we're also building those capabilities. So having a partner that not only brings the capacity but brings the expertise of the technology maybe not the expertise of the industry, but that's okay. That becomes a really good marriage between the two.
Dave Vellante
>> And your workload is predominantly training today?
Brandon Hootman
>> A lot of it's training.
Dave Vellante
>> Okay.
Brandon Hootman
>> A lot of it's training.
Dave Vellante
>> You see, I presume it's still growing.
Brandon Hootman
>> Well, I would say training and simulation and synthetic data generation.
Dave Vellante
>> Okay. And then do you see inference?
Brandon Hootman
>> Yeah, absolutely.
Dave Vellante
>> Do you see it crossing over at some point?
Brandon Hootman
>> Well, we've got to keep in mind Caterpillar's got a really large portfolio of machines and we want this capability on as many machines as possible. So as we work through that, there's going to be a continued workload of training. As we get the machines in the field and they have the capability on them, you'll see the inference start to come.
John Furrier
>> Everyone goes to school, has to get trained, and then the inference kicks
Brandon Hootman
>> in.
Brandon Hootman
>> That's
John Furrier
>> right.>> Brandon, thanks for coming on theCUBE and sharing what you're working on. Super awesome example. A lot of data. There's all the things we love. A lot of data, a lot of AI opportunities. Modern era's
Dave Vellante
>> here.>> Big
John Furrier
>> equipment.>> Big equipment. Who doesn't like dump
Dave Vellante
>> trucks?>> it on
Brandon Hootman
>> site.
Brandon Hootman
>> It's pretty cool. Thanks a lot, guys. I really appreciate
John Furrier
>> it.>> I'm John Furrier with Dave Vellante. Again, physical AI is just around the corner. It's developing super fast. Everyone's talking about coding agents, but the physical world, putting intelligence in the real world, that is where the edge comes in. That's where a lot more real-world activities come in. It's going to be a high-impact environment. We're doing our part here in theCUBE. Thanks for watching.
John Furrier
Dave Vellante