theCUBE + NYSE Wired: Physical AI & Robotics Leaders | Jim Kernan, Luxonis

Clips
More from theCUBE + NYSE Wired: Physical AI & Robotics Leaders

Speakers

Jim Kernan

CRO

Luxonis

play_circle_outline Introduction to Luxonis and their edge inference computer vision technology.

play_circle_outline Exploring Luxonis' OAK 4 Cameras: Stereoscopic Depth Perception Compared to Human Vision

play_circle_outline Benefits of processing data directly on the device vs. central compute or cloud.

play_circle_outline Competition in the market, with focus on processing information internally.

Info
Transcript

Jim Kernan, Luxonis

Jim Kernan

CRO Luxonis

Jim Kernan of Luxonis joins theCUBE’s Robotics and AI Media Week, held at the New York Stock Exchange. Luxonis makes significant strides in AI and computer vision technologies. Their focus on edge inference computer vision stands out in the market, allowing machines to perceive the world almost like humans without relying on cloud or central computing.

In this engaging session, Kernan discusses Luxonis' competitive edge and innovative product line, the OAK cameras, which bring AI-enabled vision directly to devices. Hosts John Furrier and analysts from ... Read more

explore Keep Exploring

What is the biggest differentiator of what this company does relative to anyone else in the market? add

What are the differences between the OAK 4 and OAK 4 S products, and how does the depth camera feature work in relation to spatial perception? add

What is the benefit of processing information directly on a device and sending off only small bits of information? add

What are the main differences between our camera technology and our two main competitors in the stereo depth market? add

bolt Powered by CUBE AI

Jim Kernan, Luxonis

search

>> Welcome back, everyone. I'm John Furrier, host of theCUBE. We're live at the NYSE. This is the East Coast studio of theCUBE with the NYSE Wired open community. Of course, we have our CUBE studios in Palo Alto, connecting tech and Wall Street together and creating that backbone, that open network. The NYSE Wired CUBE network is all about relationships, bringing the experts to tell you what's going on. We love bringing the guests. It's part of our Robotics and AI leaders series. Got a great guest, Jim Kernan, CRO of Luxonis. Thanks for coming on. Appreciate the time.

Jim Kernan

>> Yeah, John, thank you very much. Really, really happy to be here. Thank you.

>> You guys make really cool AI, spatial vision, computer vision. Okay. Folks who follow theCUBE know that we love multimodal AI, but computer vision captures the most data. Robots will all be run with vision, that is the core input to what we see as the future, what we're seeing in the market. You guys make product, you have product here.

Jim Kernan

>> I do.

>> Before we show the product, just quick explanation, what you guys do, specialize in?

Jim Kernan

>> Yeah, absolutely. The biggest differentiator what we do relative to anyone else in the market is that we do the edge inference computer vision directly on this device. Effectively, you're able to give machines the ability to perceive the world in the same way that humans do. Generally speaking, what I say is that anything that you are able to view and process with your own two eyes, you're able to view directly on our device without any central compute, without any cloud compute. That's the major differentiator for us.

>> Okay. I don't know if we can get a picture or how you guys want to do this. Hold it up. Leave it there? Okay, we leave it there. I can see the picture on the monitor. Okay, so this is the big fat one and it's got some labels on it. I see a bunch of cameras in here. This other one here, that is a smaller version.

Jim Kernan

>> So these products are our new Series 4 products.

>> That's a nice shot, Ken. Appreciate it. All right, this is back. I see connectors. Okay, explain the products.

Jim Kernan

>> Yeah, so these are our new newest product line that we're just launching right now. The biggest difference between these two is this is a stereo depth camera so you're able to get distance perception on top of any of the computer vision analysis that you're going to be able to run on the camera where this is just a single lens. You'll be able to run 2D inference for object recognition, detection, that sort of thing. If you start to talk about robots, they're going to need depth perception to be able to move around space. These cameras process the world same way we have two eyes. If you try to move around the world with only one eye open, it gets to be a pretty difficult thing. The baseline between the two cameras mimics what we have as humans. So again, you're able to move around the world now navigate space with this camera, with this one, and actually both of the cameras, you're going to be able to detect it and truly understand the physical environment around you.

>> What's the name of these products? Is there a product name?

Jim Kernan

>> So this is the OAK four and this is the OAK 4 S. So actually, OAK 4 D for the depth camera, OAK 4 S for this one, our newest line.

>> D is for depth. That's just depth of field, spatial. When you say stereo, what does that mean?

Jim Kernan

>> Yeah, so stereo is just the two cameras individually, they are each processing individual images. And again, same way humans do it, if you open and close each one of your eyes, you notice that you have different frames. We're taking the two frames from this camera, processing that with a neural network inside of the camera to be able to use that geometry to understand where objects are. The same way that I'm able to reach my hands out and pick this thing up, it's because I have two eyes that are helping me figure out where that is.

>> This one here might be like the eye of the robot, right? Like Terminator, right.

Jim Kernan

>> It's funny you say that.

>> We all know, we love Terminator references, so Skynet, but again, but this is what visualization, computer vision is the input prompts.

Jim Kernan

>> A couple of things. When I'm at weddings or colloquially, when people ask what we do, I say we make eyes for robots so you're spot on with that one. But yeah, I'm sorry.

>> Take me through how this gets built. Go ahead.

Jim Kernan

>> What I was going to say, sorry, I completely lost the train of thought. But really I say that we're the barrier, the interface between the physical and the digital world. We're taking and processing all of that physical information and allowing the digital world robots automated systems to be able to actually process and use that information.

>> I mean, physical AI, that's a great term that Jensen and everyone's using. I buy into it because physical and digital merged, I love the digital twin concept, have for many, many years. But it's been niched into manufacturing as more of a, how do I get a metaverse for a factory? But digital twin just means you're doing simulation. I take a little bit of a liberal definition of digital twins, but physical and digital now work together because the data is digital.

Jim Kernan

>> Yes.

>> Okay? Physical's where the value is. So you can almost stretch the definition, but then there's still going to be more data coming in. I'm sure. How much data you grabbing in from these systems, you got to still go through the data, do root cause analysis.

Jim Kernan

>> Yeah. Again, there are a number of different ways that you can use and process the information. People are doing things like creating digital twins, which in the common sense is just creating a 3D replica of the world so that's the common definition. We are just giving computers the ability to understand and just automated machines the ability to understand how to interact with the physical world. If you think of traditional systems, and again, what differentiates us versus other examples is that, I'll take it back a step and talk about humans because we obviously process the world through vision. 50% of the frontal cortex is designed to be processing visual information. Our eyes themselves have about 550 megapixels of information that they're gathering in any given moment and 30 to 60 frames per second. If you put that over the life of yourself, you process billions and billions of images at very, very high frame rates or a very, very high resolution. Our cameras are able to process information up to 108 megapixels, so not quite the human eye, but in very high fidelity and do that 60 to 120 frames per second. Taking all of that information is computationally expensive as you can imagine. Even just the throughput for connectors can't really handle all of that information. If you think about a traditional systems, you're going to take that data, you're going to pass that to some central compute or you're going to end up putting up to the cloud. That becomes, it's costly from a latency perspective, costly from a computational perspective, and then you have these central systems that require a whole lot of power so it's heavy from a power perspective. What we've been able to do with these cameras is take all of that information, have it all processed directly on the device and then send off tiny bits of information. So really all I need to know is the definition of an object and the 3D coordinates of that object, and I can send that in JSON back to a centralized compute and reduce a lot of that. We're working with some of the most advanced companies in the world as far as robotics, and they're really looking at us as, okay, that information can be processed on the edge. We don't need as much central compute, we don't need as much power. That reduces the overall payload of our system, which reduces overall cost. We're able to potentially replace GPUs and centralized compute that costs tens of thousands of dollars for something that is like 100 to 1,000 dollars.

>> It's like a DeepSeek moment for you guys. You can use the constraints, use offload here, design the system. That brings up a good question. I mean, okay, the business model, you got a nice formula there. Are you guys selling just the devices? Are you guys building fully-built robots? Where do you guys stop and where does the next step take over?

Jim Kernan

>> Yeah, we describe ourselves as a hardware company that builds software. We make our money by selling these devices like singular transactions of the individual item itself. But we have this whole ecosystem that's built around our camera, a cloud application layer that helps with fleet management and deployment. We are able to capture data from the real world to help retrain models because if you talk about the AI world, obviously there's initial training, initial development of models, and you're putting those out and those aren't going to perform in a perfect state in any individual environment so you need real-world training data to be able to operate your robots in your systems. Our cameras give you the ability to take that data from the real world and bring it back into your model and retrain.

>> Are you guys doing anything else besides just the cameras providing?

Jim Kernan

>> We do have a cloud application layer that people can use to-

>> That you guys built?

Jim Kernan

>> Yeah, we've built that, exactly. Our business set at core, again, we create the hardware. We don't require anyone to use the other bits of the software, same as your laptop.

>> So you have platform. So the software, hardware is the key secret sauce for you?

Jim Kernan

>> Correct. The secret sauce is the firmware that we've built. Firmware is extremely, extremely difficult.

>> It's got to be small and fast.

Jim Kernan

>> Everything that we do is open source with the exception of our firmware and a few calibration procedures that we have in order to do like the sensor fusion. That firmware with the sensor fusion, being able to put that on a singular device.

>> Yeah, it's smart. It's your local property and the open source gets you that integration open source vibe where let the marketplace innovate on top of or with you.

Jim Kernan

>> Exactly. We've had a number of people who've used us for very basic use cases like a webcam. They just like the ability to use our cameras, which they're AI enabled cameras, but they're like, "Hey, it's easy to program and I have control over all the sensors, so great, I can do that," and they want to use it just for that sole purpose.

>> If we build out our studio, maybe we can get some input on how to get smart cameras. If we had this deploy, we'd be seeing all the trades going down down there.

Jim Kernan

>> Absolutely.

>> Zoom in on the traders.

Jim Kernan

>> There's a number of different ways that that can be worthwhile. As you start to see a different activity of people's hands, you could do region of interest focus, and as soon as someone raises up their hands, just have the camera zoom down on them.

>> He's going and typing the ticker symbol.

Jim Kernan

>> Exactly.

>> He's trading right now NVIDIA option, put April 4th call.

Jim Kernan

>> We do have some pretty powerful sensors. We don't want to get in on the trades that they're actually executing, but some of that stuff can be done.

>> The NYSE will shut us down as fast as we can.

Jim Kernan

>> So here's actually another benefit, you could run something like that. And especially as you start to think of other markets like Europe, GDPR, privacy concerns are a very significant thing. You can take any of the information that you're receiving from our device and block it out immediately on our device so it never gets back to any other system. Again, if you have a connected infrastructure where you have to pass all of the data from one step to the next, you're invariably going to be passing PII as people are in an environment. Our device, you can blur out people's faces, you can, I mean they cast it and screen-

>> I mean, governance and policy is critical. That's a software innovation. Again, actually your point, your innovation enablement is whatever the developers want to do it. If I wanted to do a funky CUBE set up with computer vision, why wouldn't I?

Jim Kernan

>> Yeah, exactly.

>> I need it, but I have to code it.

Jim Kernan

>> We have a really great use case and we span so many different industries. That's the most interesting part about my job. We're in agriculture, manufacturing, warehousing, robotics, retail, healthcare. I'm going to leave off a whole bunch because the list goes on.

>> Computer vision is the killer app, but then the data access, voice activation is going to be key. I always say stories drive movements. Give some stories that you guys are working on, share some stories of how it's deployed, some people's benefits they've seen from it. Give people a taste of the value proposition.

Jim Kernan

>> Unfortunately, most of our customers are under NDA. I'll talk about loosely around some of that.

>> You can anonymize it.

Jim Kernan

>> Yeah, of course. We're fortunate we work with over 40 of the global Fortune 500 and a third of Dow Jones Industrial Average are customers of ours. They use it in a wide variety of use cases from navigation of robots, just machine guarding. Unfortunately, the world has gotten to a place where we are actively building machines that kill people. I know that sounds a little bit strange to say, but when you think of military, large industrial equipment like military, cars for that matter, all of these things. We have people who've used us for great applications just in machine guarding to recognize when a human is close to an object or a machine that can be tested.

>> Safety is critical.

Jim Kernan

>> Exactly.

>> Safety first is the vibe.

Jim Kernan

>> We have a great use case where one of our customers is, they're the security gate in a lot of live in-person events. They use our camera to identify the individual who's been detected to have a weapon on them. Once the weapon gets detected on them, they use our cameras to attach that detection to the individual's face so security can immediately go to that person individually. So that's a-

>> That's a real time assembly situation.

Jim Kernan

>> Yeah, exactly.

>> That's a generative response.

Jim Kernan

>> Yeah.

>> New data comes in, process on the device.

Jim Kernan

>> Yep.

>> Calls some code, the latency might be not as superfast on the back end, but you want that front end latency so it's all balanced.

Jim Kernan

>> Yeah. Again, the benefit of our device is exactly what you're talking about, that low latency, high compute power, being able to do that directly on device. Anybody who is an engineer, if you know C++ or if you know Python, you're going to be able to be a computer vision engineer, with a little help from building the models and that stuff. But you're able to access that where traditionally-

>> You're sitting in JSON too. That's no problem.

Jim Kernan

>> Yeah, exactly. You can send any bit of data, if you want to send an emoji off of it. This thing is a computer. It's probably as powerful as this laptop. You can send any bit of information you want off of it. It's just about the queues that we want.

>> All right, talk about the competition, how you guys rate, because that's the hot area. Obviously, computer vision is a killer growth area.

Jim Kernan

>> Yeah. Realistically, we have two main competitors. One is Intel RealSense, they create a stereo depth camera, and the other one a Chinese-based company also building stereo depth cameras. The main differentiator and the true difference between our cameras and is that we are able to process all of this information directly here. We're not sending it somewhere else and so, yes, we're in the stereo depth market and as are they. It's a critical component for any piece of technology that's going to be utilizing computer vision. But us having being able to bring your own model, load it on the device, you can do anything you want with it. So in that sense, we feel like we're in a class of our own. Obviously, there are other systems, larger systems that, obviously NVIDIA is a huge player in this space, but.

>> The demand for cameras is going to be off the charts.

Jim Kernan

>> Yeah, and we-

>> Smart cameras basically, they'd be smart cameras.

Jim Kernan

>> Exactly, and we pair nicely with the Jetson as well, right. There's going to be heavy loads for central compute as you have these large distributed systems that are complicated systems handling huge amounts of data, us being able to trim that down into something very, very simple, makes all better for central compute.

>> I mean, I'm truly curious, why I asked earlier about theCUBE, I mean are these video, would we be able to use these for some of our product? They're high quality.

Jim Kernan

>> Absolutely.

>> Multiple. Maybe over driving it. We'd be buying a GPU to use Excel. Is that over-driving it or is there a use case for theCUBE?

Jim Kernan

>> We have Oscar-award-winning directors using our camera.

>> For films.

Jim Kernan

>> For films, for special effects ultimately. So by no means a stretch for you-

>> We're not there yet for our talk-style format, but we could probably do some clever stuff here, I mean, I've got a saying.

Jim Kernan

>> There's always something to aspire to.

>> Jim, thanks for coming on. Give a plug for the company, what you guys are doing, the pitch, what your focus is right now, and just some numbers, share some data, give a plug.

Jim Kernan

>> Yeah, absolutely. Right now is actually a good time for us to be on the show, so I really appreciate it. We're actually raising our Series A right now, which will be a $35 million round. From the early conversations we've had with people, they're commenting quite humorously is usually people are asking us for 35 million to build what you guys are building, and you've already done that and now you're asking for that money.

>> Good validation.

Jim Kernan

>> Yeah, good validation. Obviously we need to get over the line there, but yeah.

>> Do you see any patents on some of this stuff?

Jim Kernan

>> We have some patents within our calibration process and recalibration process. A common problem within cameras is that they're going to be out of sync.

>> Yeah, we know that.

Jim Kernan

>> The physical world itself, as you apply heat to things, it's going to expand or contract the actual materials themselves. As you start to get into stereo depth, and it's very, very fine precision to be able to have these things function correctly. We have this process called dynamic recalibration where we're able to dynamically recalibrate on the fly. We just filed for a patent for that, which basically means our cameras are going to be able to operate in a wide variety of environments and a wide variety of conditions. So yeah, we create these fantastic edge AI cameras. They're used all over the world in dozens of different industries and dozens of different use cases. We're saving people's lives. We're enabling-

>> Great military application here. Tactical edge.

Jim Kernan

>> We have a number of people who are looking at our cameras for military applications, obviously can't talk about those. But yeah, again, that's the fascinating thing about what we've done is we've basically created this blank canvas that of hardware that people can build around and we have a software layer on top of that that makes it easy to do all of these things. We're really excited about the number of opportunities going through the fundraise. And really, our mission is to enable robots and automated systems to outstrip and outperform humans at all tasks in the physical world. We are enabling those systems just by giving them vision.

>> Jim, great to have you on theCUBE. Thanks for coming on, appreciate it. Welcome to theCUBE and NYSE Wired, part of our CUBE finance focus on Wall Street connecting Silicon Valley. I'm John Furrier here. All the action just to trade, the market's upside down today. We're watching it very closely, we'll have a report on that, but again, we're on top of the floor, behind us all the action here at NYSE. Thanks for watching.

Jim Kernan

>> Thanks, John.