Black Hat USA 2025 | Mike Nichols, Elastic

Clips
More from Black Hat USA 2025

Mike Nichols

GM of Security

Elastic

play_circle_outline Adapting to AI: Enhancing Security Analyst Roles and the Importance of Specialized AI Models for Task-Oriented Solutions

play_circle_outline The necessity of balancing automation with critical human decision-making in cybersecurity.

play_circle_outline The future of AI and its role in enhancing analytical thinking in security professionals.

play_circle_outline Encouraging a shift from traditional mindsets to innovative approaches in security analytics.

Info
Transcript

Mike Nichols, Elastic

Mike Nichols

GM of Security Elastic

At Black Hat USA 2025, Mike Nichols, VP of Product Management at Elastic, joins theCUBE’s Jackie McGuire for a candid discussion on the state of AI in cybersecurity and the realities behind the “agentic AI” buzz. Drawing on more than a decade of Black Hat experience, Nichols unpacks how the hype cycle is reshaping vendor messaging and why security leaders must focus on transparency, traceability and fit-for-purpose models over one-size-fits-all solutions.

The conversation dives into the practical applications of AI in the SOC, from reducing burnout b... Read more

explore Keep Exploring

What roles are evolving in the context of AI projects, and what challenges do professionals face in adapting to these changes? add

What is the perspective on the role of AI in enhancing capabilities within a Security Operations Center (SOC)? add

What are the potential impacts of AI on the field of security and the diversity of individuals involved in it? add

What impact is AI expected to have on the roles of tier one analysts in the workplace? add

bolt Powered by CUBE AI

Mike Nichols, Elastic

search

Jackie McGuire

>> Welcome back to Black Hat 2025. I'm Jackie McGuire, practice lead, principal analyst, best security host at theCUBE, and I am joined by one of my favorite guests. I'm so excited. Mike, Mike Nichols, VP of Product Management at Elastic. Welcome back.

Mike Nichols

>> Thanks for having me back. I had a great time last time. Glad we get to do it again.

Jackie McGuire

>> Yeah, you were my favorite interview at RSA.

Mike Nichols

>> Oh, stop it. - It was so much fun. And we were just

Jackie McGuire

>> talking before we started recording, and I think this is going to be a good one too.

Mike Nichols

>> Yeah, yeah. - So how has your Black Hat experience been?

Mike Nichols

>> Well, it's been hot.

Jackie McGuire

>> It's been fun... I think this is... I was trying to -

>> Not like Paris Hilton, "That's hot."

Jackie McGuire

>> No, no, like sweating my-

Mike Nichols

>> I need to change my shorts again hot.

Jackie McGuire

>> Yeah, sweating a lot hot. Yeah.

Mike Nichols

>> I think it's like my 14th year or something like that of the Black Hat DEF CON thing, whatever. But I think it's good. It's funny to see... I think there's people that are at VistaPrint or something redoing their banners. Everything is agentic now, it was AI last year, now it's agentic AI something this year. Wondering if people really know what it means when they're asking for it. I would hate to be in the customer's shoes right now trying to hear the reality. I did a few places, "Hey, can you give me a demo? " "Well, no, but here's what we're going to... " So I definitely feel for-

>> "Well, we mostly just have a chatbot

Jackie McGuire

>> that can tell you what's in our docs."

>> Yeah, yeah.

Mike Nichols

>> So I mean, there's definitely some good stuff out there, but I think always, you have to sift through it to figure out where there's some really beneficial things that are in the environment.

Jackie McGuire

>> Yeah, it was a little disconcerting the last several months to see companies just explicitly change their name to Something. ai, or somebody in an earlier interview was equating it to the whole crypto and blockchain thing, and all the sudden everybody was a blockchain.

>> Right. Oh yeah, yeah, yeah.

Mike Nichols

>> NFTs were everywhere, and they don't... yeah.

Jackie McGuire

>> Yeah. And so I agree, and especially coming into security from a data science background, I am happy that we've made Agentic AI a thing because I think it means that people have finally accepted that LLMs are big and expensive and full of trash data from our trash internet, and probably not the best way to solve 99% of problems. So for me, I was always like, "We need to understand that there are more models than this very inefficient language model. " And I think Agentic is great because then you can have regression models and deterministic models and probabilistic models all kind of in their own little space.

Mike Nichols

>> Well, and the right one for the right tasks. So you're not just trying to do this one size fits all, "Hey, we give you an AI model that works for everything, we promise. " You can select the right one for you.

>> Yeah. But who manages that?

Mike Nichols

>> Well, I mean, it's part of the new role.

Jackie McGuire

>> I think security analysts have always had to constantly shift their roles. They became security engineers as well. And now potentially maybe a little bit, I won't say data scientists, but definitely understanding the...

Jackie McGuire

>> Oh, I have a friend who is probably one of the most talented security engineers I know who has been moved to dev in an AI project because they were, everything they were releasing had some type of hole or flaw or issue.

Mike Nichols

>> Oh my goodness, yeah, yeah. - Again, being a data

Jackie McGuire

>> scientist who wrote really, really awful insecure terrible Python, I identify with the data scientists struggling right now because it's just not their strong suit.

Mike Nichols

>> Yeah. Yeah, we definitely threw people without the domain knowledge into a problem and that's part of it. It's always harder to train somebody on the domain side, which is sort of that Malcolm Gladwell's 10,000 hours of expertise versus the tech side.

Jackie McGuire

>> So is it going to be 10,000 epochs of expertise now?

>> Yeah, I don't know. Yeah, 10,000 tokens of expertise.

Mike Nichols

>> Yeah. So for anybody who doesn't know, an epoch is

Jackie McGuire

>> how long it takes you to make it fully through a training data set. So it was a really funny joke. It was a really funny joke.

Mike Nichols

>> That was a joke. I should have paused for a laugh, I apologize. We'll hold for a applause.

>> You're making me look bad, Mike.

Mike Nichols

>> No, but I think I do fully believe in AI.

Jackie McGuire

>> I think it, like everything else, it's just another tool.

Jackie McGuire

>> Python. - I mean, machine learning did a lot to help us,

Mike Nichols

>> but also it didn't save the world. AI, I think similarly, there's a lot of benefits to it, but I think we see people that are putting way too much... Numerous talks I've heard this past week has been about like, "Oh, the tier ones will go away, or the people-less SOCs. " And I think that that's, I don't know, eliminating or trying to automate away what is the most critical part of the process, which is that decision. Is this really going to impact a business? What should I do next? I don't know... I don't believe in that at all. I think that we believe AI can help eliminate a lot of the crap work that's in a SOC. I used to have to spend days collecting data or pinging-

>> Sifting through VPC logs when nobody's .

Jackie McGuire

>> Or pinging the NOC to be like, "Can I please

Mike Nichols

>> get access to this information? " Those kind of things. But I don't think you should remove the analytics that are there, the analytic power.

Jackie McGuire

>> Well, it's not like our adversaries are going to completely replace themselves with AI. They will accelerate what they're doing with AI, but you always need a mischievous kid in the loop to break stuff.

Mike Nichols

>> You need somebody passionate about the problem.

>> Yeah, so you do need AI to combat AI

Jackie McGuire

>> because that's the only way you're going to achieve the volume and scale of defense you need. But to combat the mischievous kids, you're always going to need another mischievous kid.

>> Exactly. - At the end

Jackie McGuire

>> of the day is security's just a whole bunch of kids

Mike Nichols

>> who don't know how to follow rules.

Mike Nichols

>> It is. It is. Yeah, it really is sort of just a battle of wits. But the thing I'm actually really excited about for AI and I'm pretty passionate about is that, security got relegated to IT early on. It had all this IT mess around in it, and that sort of narrowed the people that could be exposed to it and it got very much a samesie walk around here, a very much samesie of who is in security. I do think that what AI can do, it eliminates the techness and the venderness of what it is, and you can just get really great analytical thinkers into the process. One of the best SOC analysts I ever met was a meteorologist who just kind of understood weather patterns and brought that data science in. I think we can get more of those people and really break the barrier of who can be in the security team, which would be great, diversify the entire place. But I think actually, AI's going to allow us to get more humans involved, not remove humans. I don't think that's a good concept to follow.

Jackie McGuire

>> Yeah, and there's a difference between eliminating tier one analysts and changing the expectations of a tier one analyst.

>> Exactly, exactly. What do the tier one analysts

Mike Nichols

>> do? Yeah, exactly.

Jackie McGuire

>> So maybe you don't have to go gather a whole bunch of data and know six different query languages just to start the first step in responding to an incident, great. There's a great, I think it's a book called Bullshit Jobs. And so that's the thing is that that job just becomes a job because you can automate it away. But that's great because I think we had talked about this, burnout is the number one issue with lower level analysts because you're sitting in a room 8 to 10 hours a day, it's all of these alerts, they're all marked critical, you can never get through them. And it's just this-

Mike Nichols

>> And there's constantly some new tool coming in that promises to solve it that you have to learn that as well, and it is. I mean, when I was in the SOC, definitely, I would see people that are just like, " I'm so tired of this. I can't..." And you're just overwhelmed. So how can you, like I said, do the fun stuff. It is really fun to battle that other mischievous kid on the side of the world, but it's not fun to do that part of the job.

Jackie McGuire

>> Yeah, when I was talking with one of my coworkers about this, I was like, "Well, do they let you use the backup camera when you take your driver's test? " And he was like, "What?" And I'm like, "Well, we have backup cameras now, that doesn't mean that you don't need to back up into a parking space. " And they can't eliminate them when you take the driver's test, so it didn't change, it didn't eliminate the need to be trained how to drive, just it's a different expectation for your skills.

>> Yeah, things evolve, the world evolves.

Mike Nichols

>> And I think sticking a Tier one analyst into that role that it's been for decades, it's time to evolve. They won't go away though, that's the key piece. And I think it also allows us... When I was in the SOC, and I think it's still a case in many places I go and talk to, they end up really training these vendor experts. It's like, "Oh, here's the endpoint expert, the network expert," and what you're doing is really empowering them for their next role or to go to that vendor. All they spend their time on is that syntax, that rule sets. But what about learning about the adversaries that target your environment? Or are you tracking the most recent threat resources out there? If you take away that mess, they have more time to do that. They can go read threat reports, they can go understand what's happening, and they'll be much better at doing that serendipitous dot connection when they're actually looking at a threat that's in the environment.

Jackie McGuire

>> And I think once we've advanced to a place where MCP is pretty ubiquitous and we've fine-tuned these these tools to, if I'm an analyst, I don't have to turn to the tier two or three behind me and ask them a question, I can ask my assistant. And that's what we were talking about is right now we really need to see these things as kind of like interns, assistants. They're augment, they're a resource, but not necessarily something that can be fully trusted to be outsourced.

Mike Nichols

>> Yeah, they're not the managers, they're the... Yeah, yeah.

Jackie McGuire

>> Yeah, I mean, as you're... You're vice president of product management, so as you're building products, how are you trying to approach messaging so you're not just another person on the hype cycle?

>> It's so hard. I mean, the big thing we're focused on is

Mike Nichols

>> openness and transparency first. So for example, we just did a talk where we showed, you can get to the full trace of what it's doing. We actually talked through the steps of what... We love agentic because of all the times you don't even know that a vendor has released a new API to do a new thing, but it's keeping track for you. So when you say, "Hey, can I open a ticket," and whatever, it might be a better way to do it that it will determine for you in that path, that's great. But show me the steps you took, show me why you chose it, embedding citations constantly, so everywhere the assistant talks back to you citing where it came from so you can validate your work and also so you can learn more. I think that's a big piece of what we're trying to do. We are not delivering this secret, you can ever inspect it model that no one can know what it does, it's very... In fact, you can bring any model you want, it's more about the application around it and then again, the traceability, the openness of what it does. And the other big piece is we, although we have a chat, we think chat's cool, but chat's not the way people operate. Like in the SOC, you didn't go and open a chat to do something, it sort of breaks the workflow. And so we look at how we can use agentic or we can use these AI agents in processes. Can we automatically be helping to prioritize, not eliminate all the alerts, because they might be useful, but prioritize ones that say start here, or definitely on things like data onboarding, which always sucks in a sim. Can we say, "Hey, instead of you thinking about... " Hey, we built out the pipeline for you. That's just stuff that just wastes time. And putting that into the key workflows, and that's, I think hopefully we're sifting to the noise that way of saying, definitely security, bring your own model. We were fully disconnected, we'll work on... Anything that's there, we can manage that. And then being open about it, here's exactly what we're doing. You can see the steps. We're not trying to say that it's some kind of magic .

>> Yeah, the transparency is the thing.

Jackie McGuire

>> And that's one of the other reasons that I hated LLMs when that was the big thing a year or two ago, is that I've heard insanely high level tech leaders refer to generative AI as a black box, and I die inside when that happens because it's not. And I went on this rant that I posted on LinkedIn a couple weeks ago about how we should stop calling them hallucinations because when you say it's a black box that hallucinates, you're insinuating that it made up something that wasn't in the training data and spit it out. And that is not the case, it's just that LLMs were trained on really trashy data from the internet. And we were talking about stack overflow before we started recording, when I read an answer on Stack Overflow or Reddit-

>> Or Reddit, yeah, exactly.

Jackie McGuire

>> Yeah. - ... or GitHub, I go look at

Mike Nichols

>> that person's other answers and other contributions, and I assess the quality of that answer. And we'll we get to a point where maybe AI can do that possibly, but these LLMs don't know the difference between satire, sarcasm, stupidity, intentional misinformation, well poisoning. There's just so little about LLMs that I think people understand.

>> They don't make... They're not inventing things,

Mike Nichols

>> they're just regurgitating in a more summarized way the information they're fed. And that's why another part that we believe firmly in is that, although you can train a model if you want to, it's so expensive. We really believe in RAG, Retrieval Augmented Generation of bringing that context in. Because what you do is you say, "Hey, first look at my context because I know this, I know this is accurate. And then look at what the rest of the general information is and give me that too, but then cite both. Tell me where you got which answer from."

Jackie McGuire

>> Yeah, RAG with guardrails is great.

>> Exactly. - Just RAG in general where you're like,

Jackie McGuire

>> "Google this for me and then...

Mike Nichols

>> " And the problem is because it translates it back into a language that's supposed to be as believable as possible, because the discriminator of the generative AI model is, "Yeah, I'm 95% sure that's right," and it sounds so accurate and believable that unless you actually go to the sources and read them and understand, "Oh, it said this... It mischaracterized this in a very believable way. " So yeah, I think age agentic is definitely the way we need to move for that. What's your advice for your customers when you talk to them? Because I assume you have a lot of customer feedback loops as you're building product, so what's your advice right now for them in terms of what they should be investing in and where they should be looking?

Mike Nichols

>> There's sort of two camps. So the first question we typically get is sort of the people who are a bit fearful of AI. And like you're saying, what to do. And the first thing we say is, you need to find a way to embrace it in your organization because if you don't, you're just going to create this shadow AI world, where everyone's going to be pulling out their phones and it;s-

Jackie McGuire

>> I cite that all the time. I'm like, "You have no way to monitor data exfiltration that somebody looking at their laptop and typing into their iPad, you can't monitor that."

>> Yeah, and like you were saying earlier,

Mike Nichols

>> because these tools exist, why shouldn't we encourage people that are using coding help or other areas like my son's in school and even though they're not supposed to, using AI to help with homework, that's the future, why won't we teach them how to do that? Let teach them how to do that the right way with guardrails, cited. I mean, this was when Wikipedia, "Oh, don't ever cite Wikipedia," years ago. You had to know to look at the data, to verify, to validate.

Jackie McGuire

>> Yeah, because the Encyclopedia Britannica's from 1982 in my hallway, were more acc-

>> Oh my gosh, it was so many...

Mike Nichols

>> It was copy edited by two people and never looked at again.

>> Yeah, they were way more accurate than Wikipedia.

Mike Nichols

>> Yeah, yeah, it hasn't been updated in 45 years.

Jackie McGuire

>> And I think, to me, that's the first question. And I tell them, if you put a policy in place that says don't use it, they're going to use it. Find a way to embrace it, but embrace it in a way where it's easy, like you said with guardrails, that's sort of step one. And usually that's like finding a tentative version they can operationalize for their team members and things like that. And I think it's very successful, because it helps in so many areas -

>> A safe sandbox environment.

Jackie McGuire

>> Yeah, exactly. Just be safe,

Mike Nichols

>> but you have to implement something. And then I think it's security, it's just get through the hype pretty quickly. Don't believe that it's going to be the silver bullet or whatever analogy you want to put to it, but it can do really helpful things. But I think the core thing there is whoever you're looking at, whatever vendor you choose, can they show you the work? I think, no matter what, it's the same thing with machine learning, it was when I used to work with companies to say, "Oh, the model's proprietary. We can't show it to you or explain it to you. " I'm not going to trust it, right? So just tell me what it does and how, and then let me try it out. I think that's the key thing. Transparency on the one side and then you have to embrace it because it's being embraced, like you said, one by the adversary and two, they'll just do it themselves. They'll just go around.

Jackie McGuire

>> Well, that's fantastic advice. Mike, thank you so much for coming back.

Mike Nichols

>> I always have a great time here. Thanks for talking to me and talking about all the fun stuff at Black Hat.

Jackie McGuire

>> I always enjoy a good rant before lunch.

Mike Nichols

>> Yeah, why not? - All right.

Mike Nichols

>> All right. - All right, stay tuned everybody.

Jackie McGuire

>> We'll be back at the Mandalay Bay

Jackie McGuire

>> for Black Hat 2025 in beautiful, sweaty hot Las Vegas, Nevada. I'm Jackie McGuire. I'll be right back.