Name: Nick Mann, Deepgram
Uploaded: 2026-09-29T17:25:00.000Z
Duration: 16 min 6 s

Clips
More from Dell Technologies AI Leadership Symposium 2026

Nicholas Mann

Staff Technical Program Manager

Deepgram

play_circle_outline Scaling Deepgram Voice AI: STT, TTS & Voice Agents with Dell's Massive Compute, Data for Training and Serving

play_circle_outline Enterprise priorities: low latency, high performance, scalability, and production-grade reliability.

play_circle_outline Specialized AI for Air Traffic Control and Emergency Services with Continuous Retraining and Synthetic Data

play_circle_outline Modular AI Marketplace Assembles Voice and Native Models for Edge, Laptops, and Air-Gapped Devices

11:00

12:59

10:43

Info

Nick Mann, Deepgram

Nicholas Mann

Staff Technical Program Manager Deepgram

John Furrier

>> Hi, I'm John Furrier with theCUBE. We are here in San Jose for Dell Technologies Symposium in the Bay Area, of course. There's a New York event happening in October there as well. Dave Vellante will be hosting that. Nick Mann here, senior staff technical program manager at Deepgram, a company we've covered on theCUBE multiple times with Scott, the CEO, doing some incredible work around models targeted for specific use cases. Thanks for coming on. We're here with Dell. Good to see you. All right. All right. Explain what you guys do. I want people to understand at least the context for the company.

>> Yeah, there's many corners of AI, right? So Deepgram focuses on making AI understand and communicate through voice, right? So we are a core infrastructure developer. So speech-to-text models, text-to-speech models, voice agent capabilities that allow developers and businesses to use a more natural input, which is our voice, what we're doing right now, and maybe get them a little bit away from screens and clicks, which can be a little less efficient, but also just seems to be the way people are going.

John Furrier

>> I was joking with Scott, like, hey, maybe get theCUBE version of me, get that out there, do some AI for our interviews, for our intros, etc..

>> Closer than you think, man.

John Furrier

>> It's closer. I know. We're looking at— AI could probably be rolling up all of our programming for us. But we like to get the content. Explain the infrastructure piece because you guys are serving a lot. You have your own model. There's a real deep technical bench on the model side, but also you got to power all that. You got to feed the beast.

>> Yeah, yeah, absolutely. There's— I like to, not jokingly, but realistically, I have two big parts of my customers that I serve. There's an internal component and an external component where Dell plays heavily into all of that serving, but those models, those foundational models that we create and train are done by an incredible research team. That's our big superpower is we have 45 of the smartest researchers you've ever seen who can go prospect and find amazing things. You know what they like to eat? Compute. Just tons of compute, tons of data. So that's one of my customers. And then there's all the customers that subscribe and use our APIs and our models. And those are tens of thousands of our customers. Big and small, some probably going to be here today, and their workloads scale. And we need partnerships that we have with Dell to be able to scale our infrastructure both on the inside and on the outside.

John Furrier

>> You know, one of the things I'm going to be talking about here in my talk is sovereignty. And the premise that we're starting to see in the sovereignty world is there's kind of a dependency to it. You got to deal with chips, the architecture.

>> Yeah.

John Furrier

>> So if you look at the semiconductor world, separate the model from the infrastructure, that's one. But you're starting to see memory-first architectures. OpenAI is building their own chip.

>> Yep.

John Furrier

>> Jalapeño. That's a sign that if that goes faster, you're going to see a lot more better AI factories, better intelligence, more chips being made, better factories. So the flywheel kicks in. I've kind of never seen this in the infrastructure world before. So it's a whole nother dynamic. Linux was great. You put it on a server. It was great. Now we're in, okay, got this AI factory. It's essentially an entire intelligence production system.

>> Yeah.

John Furrier

>> But you have workload on top. It could use whatever model. Talk about your reaction to that because I think this is something that is a new thing.

>> I agree. This is the hype that I remember. I don't want to date myself, but when I was a younger man and working with Tony Fadell at Nest, the IoT explosion where if you weren't building a piece of hardware that was connected, right, it wasn't important. And it took over the world as far as our industry is concerned. AI is doing the exact same thing. Our ability to scale not just the compute part of it, like you mentioned, but miniaturize it and get it to where people are, whether that's on the edge, whether that's a medical professional as they're walking around the hospital trying to just streamline their workflow. Or the really heavy-duty stuff that is fun and catches your eye, I can't believe I gave a prompt and this system just generated all this for me. Now, those are two very different use cases that use a lot of the same architecture. I think we need to do both, to be honest with you.

John Furrier

>> I have to ask you because you invoked Tony Fadell and Nest. We had Matt Rogers on. I know Tony from Palo Alto since, Apple days. We're not just here because someone, magically said, hey, do AI. The shoulders of giants— Apple, Nest, Uber. These are companies that— even Facebook, they had to build their own infrastructure. So you had the pioneers eating glass, spitting out nails. But that was challenging. Talk about— because that means you guys saw something that wasn't being served. Nest built a lot of things proprietary. We did. I covered that in the past. But we're kind of in the mode now where Yeah, we need these new things, but now we actually take less time to build them.

>> That's right.

John Furrier

>> Talk about that dynamic because you guys had to do that at Nest because there was nothing else.

>> That's right.

John Furrier

>> But now you have that same ambition.

>> Yeah, the tooling's got a lot better.

John Furrier

>> Talk about that, the dynamics now for the creators, the builders now.

>> Yeah, I still think the best idea is going to yield the best product. There's a lot of conversation and a lot of hyperbole around, well, I can build anything. You might build a really mediocre idea and you don't want to do that, right? So the skill, the experience, the ability to really hone in on a great idea, the tooling has gotten you to a place where you can build it better, but productionizing it, making it robust, the kind of things that Deepgram focuses on because we're an enterprise-first company. Latency matters, performance matters, scalability matters. And I think you have to take that in two parts. I can create, I can ideate, I can have a proof of concept really fast. And then there's a lot of work to get it to scale to everyone that could possibly use

John Furrier

>> it.I love that perspective because I always joke about the SaaS-pocalypse narrative because I think it was with a former Apple developer or engineer and he said the best quote on theCUBE, "Bad software always dies." And he's kind of got a good point, but now when you start getting into the large scale of the cloud, when you start looking at data sovereignty, for instance, I remember during the car days when I was in business school, we studied the manufacturing of cars and Americans weren't pumping out good cars, the Japanese came in, the Europeans had protectionism, the French cars, Fiat, and you had— so AI is kind of like that. It's like, hey, I'm going to use Mistral because it's a French model. So AI is kind of like— these AI factories are kind of like cars. You can have a great one or a mediocre factory. Now, the intelligence is the metric of this production. It's the cars that are being pumped out. And the metaphor is, if I build an AI factory, The goal is to pump out intelligence to be used.

>> Right.

John Furrier

>> And if it's mediocre.

>> Yep.

John Furrier

>> So people don't want that. They want the best possible system to minimize the energy and cost to pump out intelligence at scale.

>> It's a fast-moving industry too though, right? What you thought was the best thing 2 months ago, is it still the best thing today? It's the fascinating part of the world that's kind of moving around us. I think all of us that are in this, that are watching this happen and working in it, I wouldn't say we're surprised, but I would say there isn't the firmest of footing necessarily. And you have to be okay with that.

John Furrier

>> Yeah.

>> Which means, you know what, what you made yesterday, that's really great. And a lot of people here use our models. You mentioned it.

John Furrier

>> Yeah.

>> Company's been around for 13 years. They took all the beats, all the tough times so that now we have all that institutional knowledge and we know what we're doing and we can create a model for air traffic control that's really good with that background noise. Right. Or safety and EMS systems where we know there's a lot of background noise, we can train it. We need a lot of compute for that and we need to be able to really get good data in, right, to be able to train on that.

John Furrier

>> So you guys have an intentional focused conversation internally at Deepgram around we got to keep innovating.

>> Of course.

John Furrier

>> What is that like? Because, you made a great point. we can't stop innovating because if there's going to be another model, you guys have to be up to date on that. That's the strategy. Yeah, you have to execute on that.

>> For us, it's a constant, it's a constant workload, I should say, of not resting on the fact that the model that we released a month ago is still the best. Our expectation is that it won't be. We think our architecture, our performance, the way that we actually productionize is really, really good. I won't say the best. Everyone's trying to be the best. I'm very bullish on what we do there, but the model itself can always be improved on. It was trained more than once. Let's say it was trained 10 times to get as good as it was when we released it. What's stopping us from retraining it on better data? there's a lot of things moving in the data world, as you are well aware, synthetic data becoming a thing, right? How good can we make this? We are game to see how good this gets.

John Furrier

>> But you have to make sure that your company has the best, fastest systems to keep the team innovating on the model side. And on the integration side.

>> Yeah.

John Furrier

>> What's the architecture for that? What's the first principles? How would you describe your thought process on that if you had to explain it to a friend? Hey, what's the best practice? What's the playbook?

>> Yeah, from an infrastructure standpoint, which is the question, how do we keep everyone happy and moving forward like that? We have to be invisible. Our goal is to be invisible as an infrastructure provider internally. If you ask our CTO, Adam, who's amazing, He would say our job is to not get in the way of any researcher or engineer who has a great idea or was asked to fix this problem for us or make this faster, right? Or optimize this better. And if hardware becomes an issue or scalability internally becomes an issue, we have to have a conversation.

John Furrier

>> And you gotta be always on.

>> Always

John Furrier

>> on.always on.

>> Always on.

John Furrier

>> I want to ask you something since you brought up the history, because we've been through the rack and stack days.

>> Yeah.

John Furrier

>> the old days, The phrase was, we're going to stand up some infrastructure. Now the narrative is turn on. And you start to see the rack-scale systems. They do get shipped. it's all self-contained. They're integrated engineered systems. What does that mean to you? Is that just a hyperbole, or is there really a move to a trend of less stand up and configure, more turn on and get going?

>> That's probably a question for CFOs and accountants of companies to probably talk through the cost models for both of those approaches are very different. Build your own hardware, build your own data centers with your own metal.

John Furrier

>> Yeah.

>> You're owning that car, right? I can go rent some down the street. That rental's a little more expensive. But hey, if something breaks, guess what? I don't have to fix it. It's owning a house versus renting an apartment, right? There's cost to all that, and some of it is for scale, some of it is for burst that happens in our industry. But our ability to build exactly what we want from an infrastructure standpoint is still our bread and butter. But our models also travel.

John Furrier

>> Yeah, we—

>> our business is based not only on our ability to provide our own cloud solution, but some of our customers take our models and host them themselves. In fact, a large percentage of them do, actually. And that's a big strength.

John Furrier

>> You got to run on heterogeneous systems.

>> Yeah, you have to be multi— you have to be very versatile. Right, to reach the audience you want to reach. And if your models are that good, that needs to be where you

John Furrier

>> go.Well, you guys have a great opportunity. And I was talking with Scott about this, CEO, and it's look, everyone's using voice, right? I walk around New York when I'm down there, people are talking, hey, ChatGPT, or they're using these headless systems, which basically takes the GUI out of it. Yeah. How are we doing? Are we streaming now? No, no, I wouldn't do that to you.

>> You look

John Furrier

>> great.Ray-Ban glasses that have a little screen in the glass. Take a picture. This is actually where it goes, right? So that means a lot of growth. Yeah. As you look at that growth, you have to kind of provision or plan for scale. I know we're at a Dell event, and so you don't have to be too broad, but like what's the pressure on them supplying you? Because what's changed on the vendor relationships? Because it used to be, hey, let's take you out for a golf outing and buy the servers. Yeah. Same price or, that game is over. Well, there's still hospitality, but it's not like just rubber stamping deals. What’s really pressure.

>> As a company who uses the hardware and buys it, that'll never stop. I'll still have to have those conversations of, hey, we need some more

John Furrier

>> compute.We need more

>> storage.We need more storage. I need more, yeah. So that's never going to stop. And so for everyone in the room that does that with me, please answer my calls. They always do. The great thing about a partner like Dell though, and I don't think we need to shy away from it, is they also have a huge amount of customers themselves that range from large to small. There's enterprise, there's small companies. Now that we've gotten a good enough partnership, long enough partnership, and they're using our own technology in their Round Rock data center, not their data center, but their innovation lab, we're working with some of their customers with them to try and provide some of these solutions, whether it's a small GB10, that's powering a small business and handling their IVR or a larger implementation of that, right? So we are meeting them where they are.

John Furrier

>> It's great, by the way, you highlighted your partnership on the other side, it's two ways, they're helping you out.

>> That's right.

John Furrier

>> And you're going in helping them out.

>> That's right.

John Furrier

>> This is something that John Roese, the CTO, and Ihab, the other CTO, who are both kind of co-leading the strategy, and we talked about two years ago when AI factories was all kind of being hyped up and talked about, we got the positioning we understood what the role would be, we were like, what do you run on these things? You can't just load Linux on it and be like, we're done. The software stack hasn't emerged. What's happening is the marketplace is forming the stack because someone might want to have the voice piece. It's one piece of the overall patchwork. Talk about that software stack because this is a benefit on the other side for your partnership.

>> Yeah, being flexible where you deploy, how you deploy, is huge. We spent most of this time talking about enterprise.

John Furrier

>> Yeah.

>> Okay. RTX 6000, I can find all the concurrency in the world. You know what I'm just as interested in is a tougher problem to solve. All the laptops that are now AI-enabled, that our kids are going to school with, that are starting to use this tooling responsibly. I have a daughter who's 21. I'm trying to teach her how to do it, right? I want our models on those, on the hardware to run natively. Air-gapped. That's where we want to get to. That's where AI as a whole needs to start meeting people where they are and how they're going to use it. That'll help, by the way, I think going forward, people's understanding of the best ways to use it, right?

John Furrier

>> Yeah. it's a whole nother comment. I know you got a hard stop and I got to get ready for the keynote, but you bring up a whole nother discussion that reminds me of not just hardware supply chain, but software supply chain. Sovereignty has a supply chain, land, power, the crown jewels, the data, the keys. So the laptop is a great example of where the future goes because you say, okay, air-gapped. Who would have thought you'd be talking about air-gapping a laptop? It was not on my bingo card.

>> No, not at all.

John Furrier

>> No, but it makes sense because I'm going to run my personalization on there and let an agent connect to my personal data. That's not a data sovereignty issue on the cloud. That's basically a sovereignty issue for me.

>> Everyone—Enterprise people like to say personal AI. Yeah, start there. Start with what you would do and how you would interface with something. Work back to build the infrastructure, the models, the software, the hardware, the systems that enable that. And we're trying to go up that chain.

John Furrier

>> Well, Nick, congratulations. Deepgram doing extremely well with the voice piece. Love that model. Say hello to Scott for me.

John Furrier

>> Congratulations.Thanks for coming on.

>> Yeah, thanks for having me. I appreciate

John Furrier

>> it.I'm John Furrier with theCUBE. Deepgram doing extremely well as these headless systems need voice. You talk to your chat, it does things for you. Those are the models that are powering it. They have to be accurate, they gotta be secure, they gotta be resilient. And of course it's coming down to the laptop personal device as well as the big systems that are powering those systems of intelligence, those factories that are pumping out the tokens. More from theCUBE after this short break.

Nick Mann, Deepgram

John Furrier

>> I was joking with Scott, like, hey, maybe get theCUBE version of me, get that out there, do some AI for our interviews, for our intros, etc..

>> Closer than you think, man.

John Furrier

>> Yeah.

John Furrier

>> So if you look at the semiconductor world, separate the model from the infrastructure, that's one. But you're starting to see memory-first architectures. OpenAI is building their own chip.

>> Yep.

John Furrier

>> Yeah.

John Furrier

>> But you have workload on top. It could use whatever model. Talk about your reaction to that because I think this is something that is a new thing.

John Furrier

>> That's right.

John Furrier

>> Talk about that dynamic because you guys had to do that at Nest because there was nothing else.

>> That's right.

John Furrier

>> But now you have that same ambition.

>> Yeah, the tooling's got a lot better.

John Furrier

>> Talk about that, the dynamics now for the creators, the builders now.

John Furrier

>> Right.

John Furrier

>> And if it's mediocre.

>> Yep.

John Furrier

>> So people don't want that. They want the best possible system to minimize the energy and cost to pump out intelligence at scale.

John Furrier

>> Yeah.

>> Which means, you know what, what you made yesterday, that's really great. And a lot of people here use our models. You mentioned it.

John Furrier

>> Yeah.

John Furrier

>> So you guys have an intentional focused conversation internally at Deepgram around we got to keep innovating.

>> Of course.

John Furrier

>> But you have to make sure that your company has the best, fastest systems to keep the team innovating on the model side. And on the integration side.

>> Yeah.

John Furrier

>> And you gotta be always on.

>> Always

John Furrier

>> on.always on.

>> Always on.

John Furrier

>> I want to ask you something since you brought up the history, because we've been through the rack and stack days.

>> Yeah.

John Furrier

>> Yeah.

John Furrier

>> Yeah, we—

John Furrier

>> You got to run on heterogeneous systems.

>> Yeah, you have to be multi— you have to be very versatile. Right, to reach the audience you want to reach. And if your models are that good, that needs to be where you

John Furrier

>> You look

John Furrier

>> As a company who uses the hardware and buys it, that'll never stop. I'll still have to have those conversations of, hey, we need some more

John Furrier

>> compute.We need more

John Furrier

>> It's great, by the way, you highlighted your partnership on the other side, it's two ways, they're helping you out.

>> That's right.

John Furrier

>> And you're going in helping them out.

>> That's right.

John Furrier

>> Yeah, being flexible where you deploy, how you deploy, is huge. We spent most of this time talking about enterprise.

John Furrier

>> Yeah.

John Furrier

>> No, not at all.

John Furrier

>> Well, Nick, congratulations. Deepgram doing extremely well with the voice piece. Love that model. Say hello to Scott for me.

John Furrier

>> Congratulations.Thanks for coming on.

>> Yeah, thanks for having me. I appreciate

John Furrier