theCUBE + NYSE Wired present the East Coast AI Leaders Executive Series | Cole Crawford, Vapor IO

Clips
More from theCUBE + NYSE Wired present the East Coast AI Leaders Executive Series

Cole Crawford

Founder & CEO

Vapor IO

play_circle_outline Vapor IO's innovative edge network system with low latency connectivity for AI applications

play_circle_outline Importance of latency in high frequency trading and gen AI applications

play_circle_outline Vapor IO's partnership with City of Las Vegas for AI enabled private network

play_circle_outline Deployment of GPU-assisted compute for edge applications

play_circle_outline The role of Ultra Ethernet in distributed reinforced learning

play_circle_outline Bridging the Gap: Network Building Strategy through Alliances with NVIDIA and Hardware Manufacturers

Info
Transcript

Cole Crawford, Vapor IO

Cole Crawford

Founder & CEO Vapor IO

Vapor IO, led by founder and CEO Cole Crawford, has developed an innovative edge network system that prioritizes low latency connectivity for AI and computer vision applications. Partnering with companies like NVIDIA, they aim to tackle challenges of distributed inferencing and data sovereignty. A recent collaboration with the City of Las Vegas to deploy an AI-enabled private network demonstrates real-world applications in public safety and economic development. Emphasizing the importance of edge compute with GPU-assisted capabilities, Crawford believes that ... Read more

explore Keep Exploring

What are some of the challenges and technologies involved in achieving incredibly low latency in the internet industry? add

What is the importance of low latency in trading and how does generative AI factor into this? add

What are some of the challenges and benefits associated with implementing AI-enabled security at the application layer in a city like Las Vegas? add

What lesson did Vapor learn the hard way when it comes to putting just CPU out at the edge, and what solution did they find to be more effective? add

What technology allows for distributed AI training, inferencing, and reinforced training between multiple data centers in a city that are several kilometers apart? add

What is the name of the backbone that Vapor has built, and what are some of the alliances and channel partners they have? add

bolt Powered by CUBE AI

Cole Crawford, Vapor IO

search

>> Hello, and welcome back to theCUBE's coverage here for Media Week in New York City. theCUBE's East Coast studio at the New York Stock Exchange on the balcony overlooking the show floor. We're in the afternoon of day two of three days of wall-to-wall coverage on Wall Street, with theCUBE connecting Silicon Valley from Palo Alto Studio here to our new studio. Creating a subnetwork here and building a great community of theCUBE, plus the NYSE WIRED with Brian Bauman and network. Really just been a great founders week here, and the AI leader on the East Coast. Cole Crawford is back, he is the founder and CEO of Vapor IO, an innovative company. Still a startup technically multi years in, but building out quite the incredible network edge system. Cole, great to see you again. I saw the big news about your Las Vegas... You had no news last time, you were teasing it out last time I was in town. Great to have you on to unpack the news at Vapor.

Cole Crawford

>> Yeah, thank you so much for having me back, and exciting news for sure.

>> So first of all, let's do a recap on what you guys are doing because I think it's really compelling. You've been a multi-year journey, building out what I consider probably one of the most well architected edge networks. You know I love networking and low latency connectivity, especially as the age of computer vision and AI comes in, and adds moving vehicles to any new way to get more data in off all types of content, vision particular, computer vision and video is a hard problem, and you need to have that network. So give a recap of what you guys are working on.

Cole Crawford

>> Incredibly hard problem, and I'll say it again. Most internet companies tend to measure latency in hundreds of milliseconds. We measure in hundreds of microseconds. And it's great to be on a show floor like this, where you actually get down to nanosecond trades via the matching engine. So cool to be here talking about latency. But we do build an incredibly sophisticated last mile backbone made up of a combination of our own optical layer combined with say OpenRAN-VG, with partners like NVIDIA, and customers like the city of Las Vegas.

>> ICE, the parent company and the NYSE, they're no strangers for low latency trade, obviously, they've been working the market for many years across many verticals of content and data latency kills. If you have low latency applications that are required, any lag in the system is detrimental to the performance. Also, the results now that generative AI factors latency into the quality of data it can get, this becomes really I think a fatal flaw if you miss this. Explain the reality of why that's so important. Everyone always talks about the laws of physics. Wall Street's no stranger to high frequency trading, they know you put the packets closer to the trade, you get an edge. But it's no longer how close you are to that packet, packets are everywhere distributed. This is a huge point and with GenAI makes it more important, explain your view on this.

Cole Crawford

>> I think it has been great to listen to other founders, and other adopters of both GenAI and inferencing, distributed inferencing one of the past couple of days. And our take has always been that if you're going to build out any sort of solution that has some kind of distributed component to it, there's a lot of technical complexity that goes into that, there's a lot of architectural complexity that goes into the network design of that. And the reality is training relies on, at least NVIDIA training, excuse me, is optimized for this backplane that includes this physical cable, an InfiniBand connection that has distance constraints. You're constrained to about 100 meters in that solution. There are new technologies, as an example Ultra Ethernet that is coming out, where you now have some pretty cool RDMA capabilities built into some of the Ethernet, call it cross-haul if you will, and it's not maybe forward or backwards, but across the town or across a data center. And these are the challenges that we've decided to take on, and these are typically reserved for massive hyperscale companies that have near unlimited budgets. But we are progressing forward with customers like City of Vegas, where both a computer vision enabled 5G network is attached to an incredible middle-mile backbone. And through that you solve not just latency issues, but you solve the economic challenges. So what's the easy button for a highly distributed computer vision platform across the city? Is it putting, call it upwards of 30 or $40 million of fiber optics to every smart pole where that camera is? Or do you build that on a low-latency 5G network with potentially a solar-enabled battery? So that camera is getting its connectivity through a low-latency 5G network, and it's being powered 24/7 by... yeah.

>> And one of the things that I've been personally passionate about in my role at theCUBE is educating people through our research advisory around how to deal with innovation strategies with the regulators looking over your shoulder. I was recently at IBM, they're trying to figure out how to not stunt the innovation growth by having over-regulation before innovation rears its value proposition, and it's happening now. So the advice I always say is go to areas where there's innovation, and everyone's working with some states and local governments. I got to say, and my advice to IBM at the time was do a consortium that's always a good play, bring multiple actors in. But get the innovation pace faster than regulation so that they're catching up, not dictating terms. And I bring this up mainly because the news caught my attention that the city of Las Vegas rolled out their new AI enabled private network facilitated by NVIDIA hardware, which you guys are powering. And I used Las Vegas as an example because Las Vegas is probably one of the most innovative areas. Clark County in particular of doing things. They've had 5G before it was 5G, it was actually 5G Evolution, E, which is basically not 5G. We're not going to ding them on that, good to have 5G out first. But this is where they're using it as a way to really make it a Petri dish of innovation, they're experimenting, but actually being pragmatic about it. The City of Las Vegas is the place to go because they have real needs, they have unique environment, tons of security challenges, it's the Mecca of all entertainment, F1's there. You got the stadiums there with the Raiders. You've got all kinds of activity. You got casinos, money, everyone wants to get that cash. Doors don't get locked in Vegas, it's open 24/7. So take us through this news that you have with Las Vegas, because I think that's, again, will tell us a little more about what you're doing, but more importantly how your product adds value at the application layer.

Cole Crawford

>> Sure. So first off, you got to hand it to the City of Las Vegas for being as innovative as they are. As you rightly point out, it is the entertainment capital of the world, and every major conference now happens there, at least it has some presence there at some point in the year from AWS re:Invent, to Mobile Congress, to CES, et cetera. Some of the largest events happen there. So security and from a AI enabled sort of architecture, and how you both technically of implementing that, and what it does for not just the city, but what it does for the pedestrians and the retail experience, and city services, the municipality itself, economic development, there's a ton that goes into the planning and execution of building a network like this. And it really depends on where you sit in the value chain of that economic development. And our goal is always to help solve interesting and complicated last mile problems without sometimes even... I don't want to say we don't care, but we may not know upfront the specific challenge that the company's trying to solve or the city in this case is trying to solve for. So this deployment originally rolled out as, "Hey, we own our own smart poles. We've got some fiber. We need some connectivity. And we're building solutions for our citizens and our businesses, and et cetera, et cetera, our hotels." And we've got new line of sight to things like cancer research and medical districts and art districts. And we've got these new investments that we are making as a city, and we want to be tip of the spear. We want companies coming here. We want companies investing here. And Vapor was attracted to not just the innovation but the real-world deployment, the desire to adopt these innovative capabilities and test them. To your point, increasing the pace of innovation is how you increase the pace of adoption.

>> What's interesting though is that what I like about what you're working on is that you have, and you were quoted in this article on Fierce Network, part of the Fierce Wireless group. I'll quote the thing, you call the deployment a software programmable network that can run NextGen AI centric workloads. Because computer vision is a big app there, that is a hard ingestion problem. So programmable and having this plumbing challenges, and then dealing that with a backhaul network and/or edge last mile solution is just technically hard. And you're doing that, so check. But the people who would want to run the application, they don't really care. They do care, they're the customer, they just want their thing to work. So in this case, identifying a missing child, or identifying someone who might be a criminal, or even just other cool computer aided computer vision related AI could do a lot of things. It's the body cam work. Going through body cams in real time. This is like our world now.

Cole Crawford

>> This is our world. And in fact, people don't realize, and we don't compete with this company, I'm about to mention, pretty sure they're actually NYSE listed. They are public, 90% sure NYSE. People don't realize there is a company that's been doing this for a long time, maybe not with AI, but I'm sure that's probably on the roadmap because if you're not looking at AI right now-

>> Especially computer vision....

Cole Crawford

>> especially computer vision. So there is a public traded company called Axon that does exactly this, right? If you've looked at any sort of YouTube video on police body camera footage, lower left-hand corner probably says A-X-O-N on it, $33 billion public traded company. Sensors are what's going to drive AI. I think large language models are super interesting as an interactive tool that we get to test around as a consumer of Gen AI, but really I think similar to 5G, AI is an enterprise. The money's going to be made on the enterprise, not on the consumer, and it's meant to solve hard problems. The things that AI can pattern match against versus what a human can pattern match against, we get bored, we get distracted, we have other things that we've got to do, and AI could just do that 24 by seven, and it's going to do a good job based on data in. Good data in equals good data out. Garbage in equals garbage out. So we like the fact that you don't necessarily need person in the loop looking at that video feed in order to detect a threat, or detect a robbery, or detect a missing kid, which that mechanical process has been the process for so long. And the idea isn't to disrupt jobs in Las Vegas. It's actually to open up more time for more people to do more innovative things. So I don't want to say mundane, but the tasks-

>> Well, I've debunked, I don't like this whole job's going away. I've seen that cliche been thrown around on every generation, it's just wrong. In fact, the jobs are created on the other end exponentially because of the shift of no one likes wasting time on mundane tasks.

Cole Crawford

>> 100%.

>> In fact, it's a morale killer.

Cole Crawford

>> It is. And if your job is just to sit and look at a screen 24/7, or review video footage 24 by seven, we now have computer vision that can do this. And so I think that this is a good thing for everybody. And this is ultimately for public safety. These cameras aren't invading your privacy, they're out in public spaces where you're already being filmed. Las Vegas is one of the most film public places you can walk around and for good reason, it's grown very quickly, it's got an incredible amount of traffic. Remember the barriers that were put up along the strip. There's a lot of partying that goes on in that city, and there's a lot of people that tend to walk in front of cars where there's no crosswalk. So there's been some challenges where the city is looking to optimize, how do you keep pedestrians safe. How do you enable both the inferencing side of that for predictive analytics versus how do you just determine the real time, or post situational awareness of what happened, to either do reconstruction or to, whatever happens.

>> I think you're laying out obviously the common sense case for where the apps are. Assume that we all agree that we need those, they've been unattainable in the past because of latency.

Cole Crawford

>> Correct.

>> I want to get into the announcement because I think this starts to show your hand with Vapor. I've seen the maps of your backbone, very impressive, but now you now are enabling real applications. This is again now the beginning of that. Take us through what this means for you guys. What's next? Obviously, programmable networks are key. I think GenAI can write code. Everyone's going down to the kernel level of things to get better performance. In this case, you've got low latency edge, you can always put more compute at these areas to do so. And then there's also a contract around distributing data, and putting more compute at edge points, and managing data differently. Take us through for how the architecture will extend out, what is happening? What's going on? How does Vapor continue to push that horizontal scale, and also the ability to do these domain-specific applications?

Cole Crawford

>> Well, you mentioned and rightly so, latency is a contributing factor for the workloads that will or will not succeed at the edge. But the other challenge for the edge, which we had to learn the hard way. This was a hard lesson for Vapor to learn, was putting just CPU out there also didn't really solve as much as it could have, or will solve with GPU-assisted compute. So this accelerated compute, you see what NVIDIA is doing here with its hybrid approach to things like Grace Hopper, where you have a 72 core arm CPU attached to a H200 with high memory capability. So you got CPU-GPU on the same PCB sitting, sharing a backplane capability. This is also equally important for how to take advantage of edge compute, where edge is not... And we could sit here for two more hours talking about where the edge is. Forget it. If you just think about what your latency profile-

>> We'll schedule that for another time, in this case lets stay focused here.

Cole Crawford

>> Yeah, let's assume that some edges like NYSE are nanosecond. Some edges like telco networks are microsecond, and some edges like general-purpose cloud could be hundreds of milliseconds. But the edge is also, it's not just a data velocity problem, it's a data provenance, a data sovereignty problem. And that is equally important to cities. Clark County wants to keep data in Clark County and rightfully so. Do we have the same sort of GDPR-style laws over here yet? We don't.

>> Privacy laws are out there though.

Cole Crawford

>> But they're out there, and you can expect that they will be become more prevalent over time.

>> Just pause that for a second because I think you brought up this point about sovereignty and then did locality. This requires intelligence, super intelligence if you will, not AGI. But this is why you brought up NVIDIA. NVIDIA is mentioned in your news story, the role of NVIDIA with you guys, what is that role? And talk about this intelligence need because this is one of Jensen Huang's biggest things he talked about all of last year is that he's democratizing supercomputing to the masses. I'm imagining that it's going to be smaller form factors.

Cole Crawford

>> Yes, we say the exact same thing. We take hyperscale-like architecture because a lot of it came from hyperscale built open compute, ran the foundation and brought in Microsoft and Google, and all of the people that were the founding members of OCP. And we're democratizing that hyperscale like cloud computing for these more localized data velocity, data provenance, data sovereignty based requirements. And you're right, it takes intelligence. And again, the reason for the NVIDIA partnership is it's not just intelligence on the 5G frontal, like which virtual network function is going to spin up and down to solve the eSIP reconnection for frontal, and sorry for being technical. So just assume that there's intelligence that goes into building telco networks. But it's also the intelligence of, hey, this is the network availability to me and not me, me, but the city or an enterprise. I need to geofence this traffic. I don't want this traffic leaving this boundary. Or I do want this traffic to traverse X, Y, and Z cities or regions, et cetera. So having a programmable network that you can architect to self optimize and self-heal that's the beauty of this, and you can do this over simple interfaces. You had Arcus on yesterday and I think On yesterday, the beauty of that type of architecture is you can now build constraints into what the network is supposed to do in context of the application, and that's key.

>> And we're in a super cycle right now. This is a key time for your venture. What's this mean next? Does this scale? What's your plans? Can you share? Is it too early to share? But this is already, I can see the direction, connect the dots for us.

Cole Crawford

>> Look, I've constantly been a contrarian to the general consensus of internet evolution, but darn it, I've always been right. I deployed the very first Linux box in a major telco in 1999, maybe '98. And everyone said-

>> You actually had to load the software on the machine-

Cole Crawford

>> Totally....

>> back in the day.

Cole Crawford

>> With floppy drives,

>> Young guys are like, "You loaded software on a machine." So what are you constrained about now, what do you think that you're right about?

Cole Crawford

>> Let's assume for a second that languages typically move slowly. I think the US language adds 15 words to the dictionary every year, that's not a data velocity problem. That's a training issue there. But we've trained, I think even Sam Altman said, "We've trained 99% of the world. All of the world's data we've trained most of it." So what do you do? Well, if you want to keep training, there's a lot of the world that needs to continuously be trained. And that's all of the entropy that happens out... Data gets generated at the edge. Now you and I are edges and we generate entropy.

>> We're a smart edge right here, stock exchange.

Cole Crawford

>> Exactly. Intersections are edges. Retail shops are edges. This is where data's created. This is where data should be analyzed. So there was a author, Jim Gray, Fourth Paradigm from Microsoft said, "Don't bring storage to the compute, that's super inefficient. You've got to bring compute to where the data is." And the data is now in the real world. So if you want to, I think through GenAI and kind of the magic of what that can do as you interface as a human, but I think that that is extrapolated by an order of magnitude when it becomes machine to machine.

>> This point, I think you're right about inference. You and I have talked many private conversations about inference off camera and on camera here. AI inference is the killer app because you're right. The old school model was an application that has data, the user could load data in and then they're consuming the data that they just put in or whatever. And their data was usually out band or not collected, or it's in their head. Now the human or the edge is consuming data in the application, the generative application, but also throwing off data. So you now have a flywheel of data velocity in the interaction. So now you have now a net new data set. So the reasoning and inference comes in, the reinforced learning comes in. So you're starting to think, "Okay, now what's that paradigm?" I think of the whole AI thing like school. I go to kindergarten, I graduate high school, that's training. Maybe college, put college in there. I don't go back there. I don't go back to 4th grade. I already learned 4th grade, I don't need to go back to 4th grade. I then infer from that knowledge that I've been trained on, I might go back for some classes to augment my current training. So training will always be there, but not at the bulk of maybe the Gen-1 training. But as I learn new things that infer, I am updating my overall data set in my brain, This is reinforcement. So this is the progression as edge devices throw off data, it could be telemetry data, it could be interaction data, it could be application to application, human in the loop, whatever that application and user is. It could be machine or humans. There's a new data set that emerges. I think this is where AI gets clever, because now the edge could use the low latency but also process faster.

Cole Crawford

>> That's correct.

>> Jim Gray's point, bring the compute to the edge.

Cole Crawford

>> That's exactly right. And look, you can't help, but we all have our cognitive biases. And the idea is if you can reinforce AI with data-based biases that are quantitative and not qualitative, then it's just a really fantastic... No different than a matching engine, it becomes algorithmic. And your biases are based on historical fact, and not opinion. And that's what you want out of AI.

>> So you're saying solve the bias problem downstream, don't try to fix the bias.

Cole Crawford

>> It's almost impossible to do it upstream.

>> Well, because you're starting inherently with bias. Because whatever your bias is, it's biasing the algorithm there. So you're saying as you get more data, collectively the intelligence of the super data set-

Cole Crawford

>> And then it's not synthetic data either, it's real world data that is happening at the intersection. The person walks in front the car, that was a true event, and the algorithm could be taught that that was an actual true event. It wasn't some simulated multiple of predictive analytics.

>> And that's why I believe that you're right on that because at one point this is a really good insight, is that the biases, no matter what they are, it could be gender bias that could be drawn by the application, it could be machine bias what I'm looking for, things like misinformation in the media. Over time, whatever insights you can service to refresh the data relevance drives the normalization of the data.

Cole Crawford

>> This is where Ultra Ethernet is going to play a massive role in that distributed-

>> Explain....

Cole Crawford

>> reinforced learning.

>> Explain the Ultra Ethernet impact.

Cole Crawford

>> Again, just if you're a city and you operate like we do multiple data centers around that city, and you've got an AI workload that is inferencing lots of things around that city. NVLink doesn't cut it. I can't make InfiniBand work between those two physical data centers that are four kilometers or 10 kilometers away. Ultra Ethernet gives me that Ethernet layer that sits one layer above our fiber optic layer, and it does allow for that reinforced training. So that AI model, what a beautiful thing to have to distributed training, distributed inferencing, distributed ingress for reinforced training, and then redeployment to that inference, it is awesome feedback.

>> Also, I'd add to the Ethernet to piggy-back on that and say Ethernet is open. So there's an ecosystem of vendors involved in Ethernet.

Cole Crawford

>> That's correct?

>> Interconnects are great for short distances, no one really cares what's under the covers. Does it work the way, faster? Yeah, who cares? . Great. But that's not the global scale. There's always a big fight between InfiniBand and Ethernet, but it's not about one or the other, it's about how they're used. If I'm going to collect a bunch of clusters together InfiniBand is great, because I have high speed between the two systems that are working together to do something. All right, so where do you go from here? So let's step back. You got the deployment, news is out there, you got the vision. What's on your radar? What's your plan?

Cole Crawford

>> Our plan is to continue building these last mile networks that matter for the large enterprise that has that supply chain issue, that wants to automate that next-gen infrastructure where some of the complexity is how do I solve for a wide area network at scale? Because the reality is there's no one fiber company that has coast-to-coast fiber, one continuous cable that goes from one coast to the other. There's lots of local fiber from lots of local vendors putting that together. So you can go to an enterprise and say, "Look, it doesn't matter what city you're in, here's a layer two network that's available to you where you have the most optimal latency from lots of different providers that has been abstracted away and guaranteed you're going to have the SLA that you expect." Because as you know, John, internet is best effort. If you can now offer an SLA on the network, an SLA on the application, an SLA on the infrastructure, you're now talking about solving real world problems for large enterprises, which I think is everybody's growth engine, right? Does SNB move the needle for GenAI? I don't think so. And do consumers move the needle for GenAI? I don't think so. It's large enterprise, that's where the opportunity lies. Everyone assumes that large enterprise has it all figured out, they're just going to do it themselves, but that's actually not true. Large enterprises often need vendors to solve... If that were true, Goldman Sachs, or JP Morgan, or whoever would have no need to ever even build a financial cloud in Amazon, they're big organizations with a lot of money. So big organizations also need help solving complex challenges. And when it comes to zoning and permitting, when it comes to running networks, when it comes to building accelerated GPU architecture, these are hard problems that we intend to just make easy at scale.

>> And the other thing that you're doing, and I like is that you're taking, which is a construction-like mindset of getting footprint, and building a system architecture on top of it from day one. Not just think, "Okay, I got some location, now let's bolt on a machine, got a base station," or whatever the telecom analogy would be. Great stuff. Cole, I really appreciate you coming on theCUBE with the news. Last 30 seconds, put a plug-in for Vapor. What are you looking to do, hire, plans? Go.

Cole Crawford

>> There's again, a lot of complexity that goes into building these things. We actually call this whole thing zero gap because we've left out all the gaps from tooth to tail Vapor has built one of the most capable backbones, both locally in market as well as nationally. And we've got some incredible alliances we built with NVIDIA and hardware manufacturers, et cetera. And we've got some incredible channel partners. We've been public about some of those. Do the research on the web, who our partners and our channel partners. But appreciate it John, thanks so much.

>> Great to see you. Cole Crawford in theCUBE and NYC WIRED network, obviously great to have him on. The networks are a really big part of what's going on, not just the physical layer, but as it moves up the stack around data, and understanding the system's mindset, and the system's architecture. A lot of trade-offs to be made. Latency is not one of them. This is going to be the future of generative AI. I'm John Furrier here at our East Coast locations. This is our super node, where we bring all the access from New York. We'll have a team here on the ground, building out the network, of course, connecting that to Silicon Valley, from Silicon Valley to Wall Street, wall-to-wall coverage. Thanks for watching.