AWS Mid-Year Leadership Summit 2025 | Rohan Karmarkar, AWS

Clips
News
More from AWS Mid-Year Leadership Summit 2025

Rohan Karmarkar

Director, APO Technology, AWS Partner SA

AWS

Tools and app modernization support AWS cybersecurity partners in managing ongoing threats

As the technology world has grown more complicated through an explosion in artificial intelligence use cases, it has also led to challenges for cybersecurity professionals. Generative AI applications must be secured, the infrastructure around AI needs to be protected, and there is a constant worry about new threats driven by AI-powered tools.AWS’ Rohan Karmarkar talks with theCUBEs John Furrier about cybersecurity.For major cloud practitioners such as Amazon Web Services Inc., this reality requires close coordination with its partners to ensure that these factors are properly managed

play_circle_outline Enhancing Cybersecurity: The Role of Data Flow, Observability, and AI Integration in Securing and Defending Against Threats

play_circle_outline Security is paramount, particularly for AI application development and infrastructure.

play_circle_outline Strengthening Security in Generative AI: The Role of AWS and Partners in Innovation and Incident Response Automation

play_circle_outline The AWS marketplace facilitates partner integration and business growth.

play_circle_outline Application modernization is critical for maximizing AI value in enterprises.

play_circle_outline Ongoing discussions about data architecture and integration patterns are crucial for future success.

Info
Transcript

Rohan Karmarkar, AWS

Rohan Karmarkar

Director, APO Technology, AWS Partner SA AWS

In this exclusive segment from the AWS Mid-Year Leadership Summit, Rohan Karmarkar, Director of APO Technology at AWS Partner Network, joins theCUBE’s John Furrier to unpack how AWS and its partners are driving next-generation cybersecurity innovation with generative AI. Karmarkar outlines the three primary use cases emerging across enterprise security: securing gen AI applications, using AI to improve threat detection and defending against AI-powered attacks. He highlights how AWS partners like PagerDuty, Trellix and CrowdStrike are integrating services such... Read more

explore Keep Exploring

What are the key challenges related to enterprise AI that organizations are currently facing? add

What are the key considerations for ensuring security in AI applications within AWS? add

What role do cybersecurity partners play in utilizing AWS generative AI capabilities for enhancing security? add

What are the reasons for customers and partners choosing AWS and its Bedrock platform for AI applications? add

What factors are considered in building different types of applications, and what innovations are occurring in the agent space? add

What are the key considerations and trends currently being discussed in the field of application and data modernization, particularly in relation to AI and security? add

bolt Powered by CUBE AI

Rohan Karmarkar, AWS

search

>> Hello, welcome to theCUBE here in Seattle for our special AWS halftime coverage, midpoint between re:Invent and when it ended and when it's beginning. I'm John Furrier, host of theCUBE. Rohan Karmarkar is here, Director of APO Technology, AWS Partner Network. He's a solution architect who works with partners and looks at the future with generative AI. Great to see you Rohan. Thanks for coming on.

Rohan Karmarkar

>> Yeah, thanks for having me, John.

>> So the enterprise AI is hot. The partners right now are seeing a lot of opportunities. They're getting deeper engagements, they're being asked to do more. What is the big enterprise AI challenge that you're seeing right now?

Rohan Karmarkar

>> I mean, gen AI, as you know, has a broad set of applicability and it can be applied in all industries and all technology. Security is one of the key ones in my mind. If you look at some of the cybersecurity customers and partners, they're seeing three different patterns there, right? One is securing the gen AI application that everybody's building and a lot of them are building a lot of apps. The second is around using AI for security itself to improve security engineering. And the third is defending against the new threats that are AI-powered threats that enterprises are facing. Again, so seeing a lot of interesting things AWS is doing as well as our partners are doing in that space.

>> What I love about the time we're in right now, and I love this halftime report because it's been such a fast year, it's almost a half a year is one full year in a normal kind of AWS time and cloud time. But the data side of the equation is so important for the AI and as the enterprise infrastructure gets faster, that's going to unleash more agentic and agent and more data-driven architectural changes. Now that being said, you then say, "Okay, cyber is a data problem." So cyber and AI kind of go together as I look at it as one thing. I think a lot of people are saying, "Okay, it's all kind of one thing is data flow and observability, predictability, preemptive." So data and cyber are almost like a one thing. That's why you're focused on that.

Rohan Karmarkar

>> Yeah. And the AI infrastructure as well, right? So within AWS, as you know, security is job zero. You might've heard it so many times. Because the security culture starts at the top and then it expands into all parts of AWS and it's everything that we've built, right? So when you look at AI for example, securing the AI infrastructure, making sure that the AI data, the model rates and other things that are sensitive to the customer are secured from the operator, that is us, or from within the customer's access controls are critical. Data not being exposed to the internet, that's critical. So encryption is critical and communication is also critical. So you're spot on. Data is key. Securing that data and network is key for AI applications.

>> I mean, security has got to be factored in from day one, but it's also part of the architectural design. Last re:Invent, I talked with Lori Beer, she's the CIO at JPMorgan Chase, they love machine learning. They've been doing machine learning for years and kind of early AI, but generative AI has a high resilience security bar, so production windows for getting stuff into production has always been a challenge. Your partners in the collaboration with AWS and your partners have been always critical for the success and growth. I mean, Matt talks about it all the time, Andy before. So the relationship with the partners and then the end users say at JPMorgan or end user customer is critical. So how are the partners looking at integrating gen AI into whether it's their stack or their product, what are some of the core things that you see? Is there a pattern? Is there a top three things that you see emerging that are top of mind that are kind of tips?

Rohan Karmarkar

>> Yeah, see, for partners similar to customers, security is important, right? And that's why they trust AWS to build their gen AI applications in it. Now, if I focus on the cybersecurity partners, those partners basically are building a lot of capabilities using AWS generative AI capabilities to help customers achieve the secure end-to-end security that they need. So they're complementing the secure foundation we are building and then building on top of that. So I'll give you a couple of examples. PagerDuty, one of our key partners uses Amazon Cube Business, Amazon Bedrock, and SageMaker to improve the incident response automation with AI. There are other partners like Trellix who's got Trellix-wide security assistant, which is again built on Amazon Bedrock and SageMaker. So this assistant basically uses Claude as well as Amazon Nova models to really analyze the different security incidences that they get. Typically, security engineers get overwhelmed with these incidences, so bringing in the right context and helping them analyze and getting them more productive is something that security partners are doing.

>> At RSA, I did see Trellix, I love what they're doing. I know you guys are involved with them, but they're also trying to get their product to their customers.

Rohan Karmarkar

>> Yes.

>> What's some of the feedback you've heard from partners? And I just was interviewing another partner on the ISV side, there's a slew of ISVs coming on, Deep Instinct just got a new certification, which I didn't know called Deployed on AWS, which... I don't know if that's new or not, but that's kind of like on AWS, but it's a certification. So you got ISVs and you got the Trellix of the world. What's the feedback? What's the reason why they're partnering with AWS?

Rohan Karmarkar

>> I mean, see, there are multiple reasons, right? Security, again, is critical. The reach that AWS can give them with respect to the large set of customers, that's critical as well. I think marketplace has been a really a differentiating thing for us and this is where a lot of partners use marketplace as a channel and you see a lot of partners are doing more and more business to marketplace. And on the AI side, I think from day one we always believed that customers and partners are going to need different models for different use cases. They're going to need services that can run within their VPC that can give them data isolation. So that's why Bedrock and all the capabilities with Bedrock, that includes RAG, knowledge pieces, ability to fine-tune, model distillation, all of those are really resonating with them and that's why they're looking at Bedrock as the core platform for building AI applications.

>> I think the whole Bedrock and then this distillation trend that we published, I don't know if you saw it, two years ago before the wave hit, the power law of AI and we were the first ones. And then Matt went on the record early, Matt Garman, that models will integrate and there'll be many models. So we saw that power law of big popular models and then a long tail. That's what we're seeing now, but we're seeing more of the small language models being distilled from the large language model. So why build a small language model when you can just distill off Anthropic and then serve your needs with your proprietary data? So there's a lot of strategies. You got MCP, it's very popular right now on the context protocols, similar models. So what is the architectural challenge that your partners are having right now? What's their opportunity? Because there's so much up in the air right now, it's going to come down. How do you see that coming down and landing?

Rohan Karmarkar

>> Yeah, I mean, by looking at different kind of applications to be built, like model choices is a critical factor. Things like cost, ROI, accuracy, specialized domain expertise, security openness, all of these factors are being considered. In the agent space, we are seeing a lot of innovation happening with AWS as well as with our partners. So recently out of RSA, there are a couple of partners like Panther AI and Securonix who have launched various agents that help in security engineering aspects, security operations. So we are seeing a lot of that agentic kind of workflows and applications being built now. They're starting with simple agents, but we are also seeing trends where agent to agent communication and multi-agent kind of workflows are there. So a lot of innovation and, like you said, the pace of innovation is amazing with partners, they're adopting the new kind of AI architectures and the architectures are evolving, the protocols are evolving, but there's a lot of definitely hunger for using that innovation.

>> Nandini was on yesterday, she was talking about... She's the vice president of Observability, Search and Cloud Watch to get all that infrastructure, AI ops kind of thing. That's huge data and there's a lot of data that you guys have at AWS. I'm just curious when you talk to partners, I'm a partner and we're technically in partner network, we're small, but if I was a big partner, I'd want access to the AWS knowledge base. You guys have such a plethora of data, you have all that stuff and you've got the AI tools, you've got Bedrock. Are partners integrating more in because the agent's going to do well when they have more of data? What's the vibe of the partner when they say, "Amazon, what can I do more with you on" and how do you lay that out architecturally?

Rohan Karmarkar

>> So I want to differentiate between two things, right? Customer data is completely owned and managed by customers, right? AWS does not have access to that data. As operator we -

>> But you've got your own data.

Rohan Karmarkar

>> Yeah. So we do have a lot of patterns based on how developers are building on AWS and that's all now being productized through Q. So Q is a very powerful tool in the hands of builders, whether it's Q Developer or AWS Transform, which can be used for application modernization. So like you said, one of the other patterns we are seeing is for customers and partners to really get the value from AI, they need to modernize their applications and data, right? So that's where tools like AWS Transform are going to help them modernize faster. Q Business is going to help them integrate all the different data silos and different data source that they have. So these are the kind of interesting kind of trends we are going to see with that. And a lot of that we've provided to our partners and customers through Q, primarily.

>> Yeah. Rohan, one of the things that jumped out at RSA this year, and I went the year before, and actually we've been there I think eight straight years with theCUBE, but 13 years covering it with SiliconANGLE. Last year and this year were completely different. The year before they were poo-poo, the agent's all hype cycle, security people are skeptical, they definitely can't pull BS on security folks because they a tough job. They're on the front lines and they don't put up with any BS. This year, it was, no agents are happening and they weren't over the tops hyping it. They're just more practical about it. And so what I saw was that transition from, "We'll see, I'm never going to trust that hallucination." But they're really put into work in these workflows. And so that was the big takeaway from RSA. So I can imagine that agents are going to come in and the other thing was that the SOC was changing. So because that's dashboard driven, there's a lot of work and data telemetry coming in. So the role of the SOC seems to change. How are you seeing those agents emerge in cyber? How are they translating into the SOC? How are they translating into the data architecture for a company?

Rohan Karmarkar

>> Yeah. And you're spot on. See, if you look at the security industry, there has been a deficit of security skillset. It's been an industry-wide challenge for many years now. And I think AI is helping the security analysts and security operations and overall the SOC 2 to be more productive. So we're seeing a lot of those, right? We're seeing those security assistancts being built by of our key partners like CrowdStrike, which is the industry leading endpoint security platform in Falcon, has built those assistance and then they basically are helping analysts to create events, bring in context so that they can resolve those events pretty fast. And also with agents now able to respond to those events, right? So a lot of those capabilities you started to see, right? So definitely we're seeing gen AI as an accelerant for security engineering and being adopted widely across our security partners.

>> Yeah. It's funny, we had two of your partners, founders on theCUBE at RSA, Palo Alto Networks and CrowdStrike. And it's very interesting, and Palo Alto was really interesting to me because Zee I think was talking about the platform vision. He's recognized that the current state of the platform and tools has to evolve faster because the agentic opportunities, one, there's kind of a re-platforming, number two, the adversaries are getting better.

Rohan Karmarkar

>> Yes.

>> So the whole AI helps the defenders. Yeah, that's true, but they still can be wrong ones. So the adversary is getting faster and smarter. There's more entry points and so he's essentially saying, "We have to redo it." I'm paraphrasing, but he's basically saying we need to reset, we need to redo on security architecture.

Rohan Karmarkar

>> Yeah, you're spot on. I would say, like you said, the adversaries and the bad actors are using the same gen AI tools to create novel attacks. We're definitely seeing increase in velocity of attacks. So obviously security partners have to cater to that. AWS has to cater to that. And it's not just along with our partners, our own security services, if you look at it. Like security services like Amazon Defender or Inspector or CloudTrail, all of these security services now have gen AI capability. So partners who are using this as well as customers who are using this have that power of gen AI technologies to prevent these incidences, analyze or even defend and even resolve these pretty quickly.

>> I had a great conversation with Matt Garman yesterday, the CEO of AWS as you know, and I was pressing him on this notion, I've been kind of over the top with a little hyperbole on our CUBE podcast. Dave Vellante and I were talking about, "Oh yeah, SaaS is being replaced by agents and it's going to be a 100x step function." Of course I wanted to get Matt's reaction and he actually had a great answer. He basically said politely, "No, it's an evolution." And I like what he said, because SaaS is just current state, it doesn't really go away, it's just agents are on top of it or an extension of that. And so I buy that. So it's kind of also verifies the fact that agents are coming faster and will be bigger, but that point is about evolution. So even though security is a redo, you can't throw out Palo Alto, you can't throw out CrowdStrike, those will continue-

Rohan Karmarkar

>> To evolve.

>> Even legacy networking gear and networking fabrics will still be around. So the abstraction and evolution of agents are here. How do partners figure this out? Because you're seeing partners that either have tooling or a platform want to evolve those platforms with AWS. What's the pitch? I mean, pitch but not a sales pitch, but what is the pitch for that partner who wants to sequence to the new opportunity?

Rohan Karmarkar

>> Yeah, I mean there are two things there, right? One is obviously these partners are making sure that they build capability to defend against the new AI-powered attacks and also build capability to make the security engineers and SOC more productive, more automated, right? So those are the two things that we're working with our partners to build those capabilities using the secure foundation that we have. So I think that that's the key thing and you're spot on. These solutions and products are going to evolve. They're going to have agent and agentic capabilities as a part of it, but it's not going to be completely replaced by .

>> Yeah. It's a great opportunity for you to help your ecosystem. And then obviously the moonshot project AWS has, Colleen Aubrey talked about that, but the partners want value-add because they have end user customers, which is your customer too. So the question that I would ask is how are you guys looking at that enablement? Is it technology advancements? Is it deep research? If you look at all the successes, at least we've covered on theCUBE and document on SiliconANGLE, the pattern that we see is there's real research going on, not like R&D classic applied R&D, but real technical inspection of what they have currently, where they want to go on the business model side and tech. And so there's also deep tech research like what's going on with the models, so what's the price performance of the tokens based on the infrastructure? How do we get more tokens into the app? And a little nerdy, but there's big conversations because that drives the scope. These are huge questions. So how are you guys bringing that to the table? They might be resource constrained.

Rohan Karmarkar

>> Yeah, so you're spot on, right/ All of these architectural discussions are happening everyday. Decisions have been made. So a fundamental approach there has been working backwards from what partners and customers need, what are they trying to achieve, how we can solve that problem through our technology, right? So that's the approach we typically take and we're seeing different architectural choices and decisions being made around models, around fine-tuning, around security and things like that. So those are the things we definitely get involved in. Again, we want to provide what we call the right primitives for the partners so that we can build on the primitives. And then on the co-selling side and the marketing side also, we have various AWS programs that help them support -

>> The marketplace is just that easy, no-brainer. It's a fast path to money. So that's on the sales side, but there's a lot of technical.

Rohan Karmarkar

>> Absolutely.

>> Where's the action? If you had to stack rank all your whiteboard sessions, you guys love whiteboards, I know that. Let's get on a whiteboard. Andy Jassy always used say that, "Let's get on the whiteboard." So what's the whiteboard conversations? If you've had to stack rank into frequency of things you're talking about, is it model integration, is it security?

Rohan Karmarkar

>> Yeah. So there are multiple things there, right? So we definitely are talking a lot about application modernization because application and data modernization is the foundation to have a strong AI. So there's a lot of conversation around app modernization. How do we get the apps to use the latest technologies, provide interfaces or databases to be vector enabled, to be supporting MCP and things like that? So a lot of those architectural decisions are happening on the data and the app side. Security is always critical to every customer and partner. So security also is in the picture. And then also around models and fine-tuning and , how can they use all of these kind of capabilities? We're also seeing a lot of integration patterns between different ISPs integrating with each other or ISPs integrating with AWS or multi-partner solutions coming. So there's a lot of discussion. Yeah, it's an interesting time to be in tech.

>> Integration is a huge piece.

Rohan Karmarkar

>> Yeah, it is.

>> Cyber is probably number one. I would agree that's kind of like it's in everything, so it's always at the top of the list, but it's not the most volatile decision. I mean, security is just you got to be on it, cover it like a blanket, don't screw up. But the model stuff, you could make a wrong call. If you pick the wrong architecture for how you're handling your data platform and your intelligence layer, you could be on the wrong side of history.

Rohan Karmarkar

>> Yeah, definitely. And we are seeing customers, they start with one model, they do a POC, but when they're trying to scale it to production, economics sometimes kind of hamper that. So they're looking at different techniques to optimize that architecture, which it could include distillation, which could include using more open source model. That's where Amazon, Nova, we're seeing a lot of good use cases with Nova models because they're one of the highest price-performance models out there with different sizes as well, right? So we're seeing kind partners change those models as they scale their architectures and kind of-

>> And by the way, they can mix and match.

Rohan Karmarkar

>> Yes.

>> That's the key.

Rohan Karmarkar

>> Yes.

>> No one model rules the world.

Rohan Karmarkar

>> Yeah.

>> But I think this distillation is a good concept. Honestly, production, you mentioned production. It's kind of one of my hot buttons, both on the enterprise AI side as well as the emerging startup. There seems to be, certainly in the enterprise, Kubernetes has done a great job of rallying the platform engineering, DevOps, community DevSecOps, so looking good with Kubernetes, people can rally around that. With the models, I don't see an area that people are rallying around. Do you see a momentum around what startups can rally around or ISVs and partners? Because there are companies that have great solutions right now, but the enterprise buyer, they don't know where to put them. So there's like what do you rally around in the stack? Is there a portion of the stack that people are getting excited about? Is it Bedrock, is it SageMaker? Is it something external on the standards bodies? Or what are you seeing around that piece because that's where I think it's going to fall, come down and land?

Rohan Karmarkar

>> Yeah, no, so I think it's the entire value chain of gen AI, right? So obviously, you start with the AI infrastructure to run training and inference then the models, but we are also seeing a lot of work happening around the tooling side of it, whether it's developer tooling, whether it's SDKs, we just released trans SDK which makes it easy for customers and partners to build gen AI applications using different best practices and reference architectures and standards part of that tooling kind of SDK. So we are seeing a lot of adoption of those as well because they want to build that end-to-end application, right? So I wouldn't say there's one area.

>> That's a good point. Good call out. I just want to just call that out because I think the tooling of all the coding assistant areas besides doing the heavy lifting for coding, one of the bright spots is the tooling is moving really fast.

Rohan Karmarkar

>> It is.

>> Fast and highly functional.

Rohan Karmarkar

>> Yes, 100%. I think the tooling is critical because, again, when you look at an enterprise application or they're looking at data across multiple data stores and platforms, they're looking at integrating it, they're going to build agents which are going to be more complex. They're going to build agentic -

>> Scope is the problem, and scope, the magnitude of the old way, new way to build a tool, and it's not like selling a tool, it's like a development team might have... This is what we saw in the old days, "Hey, we have to integrate with so-and-so or do this or that. Let's build a tool for that," and they have to build a tool from scratch.

Rohan Karmarkar

>> Yeah.

>> Now, it's like that would be weeks, maybe months and it's grind work. It's a lot of-

Rohan Karmarkar

>> Yeah. We're seeing another kind of interesting-

>> Unless it's strands and it just turns into a genius move. But talk about that.

Rohan Karmarkar

>> No, I think you're spot on there, right? And we are also seeing different patterns emerge, right? People two years back or even five years back used to think about, "Okay, if I have to integrate, I need to expose a bunch of APIs and the APIs will talk to other APIs and that's how the integrations would build. And so I need to expose the APIs." Now, we are also seeing some interesting trends where customers are, "Oh, instead of APIs, why don't we build some MCP servers and build... Enable that integration through agents rather than through APIs." So it's still early days, but there are definitely-

>> I mean, agents can do a lot of things you can delegate to, security.

Rohan Karmarkar

>> Yeah.

>> Not go through a black box and figure out what's in there.

Rohan Karmarkar

>> Yeah, I mean the probabilistic nature of agents definitely is important for driving some of these kind of innovations, but also it requires some sort of a human in the loop, that's what's -

>> Will there be deterministic and non-deterministic agents? I mean, you have to kind of nail these agents otherwise if they're not scoped.

Rohan Karmarkar

>> Yeah. Yes. I mean, that's an interesting thing to keep an eye on in my opinion, right? A lot of innovation in that space. Some of the managed capability around agents is also going to be interesting to look at, so.

>> So partners are very active with AWS. We're in a moment in time where if you look back in the past six months, a lot's happened. I know we see all the evidence out in the ecosystem, and by the way, we are also reporting that partners are doing more, the customers are asking them to run things.

Rohan Karmarkar

>> Yes.

>> Well, not just service and provide advisory. Actually, some of the biggest partners are deploying and managing the infrastructure. So that's a great sign for you. And obviously the work, it's a little harder working, do more, but that's good. We're in a transition. If you look back, Rohan, in the past six months, what would you say is the big thing that's happened since re:Invent and what would you say? I mean, we've had a lot of action. I mean, clouds models come out it seems like every other month.

Rohan Karmarkar

>> Yeah, two or three areas, right? We talked about AI. There's a lot of partners wanting to integrate and use AI services, whether it's ISV technology partners or whether it's consulting partners, building capabilities to advise customers and implement various AI projects as well. Security also is, like you said, it's always top of mind for CEOs and customers and partners. So that's again, a top area. The third trend which we talked about is application modernization, right? I see a lot of conversations now happening around, "Hey, we really need to take on that modernization project, which we've backburnered for many years in order to really get the value of AI." So there's a lot of conversation around application modernization that the partners are really running.

>> I think the app modernization projects are just also spawning off into full data projects because-

Rohan Karmarkar

>> I mean, app and data go together, yeah.

>> And so what it does is it says... Again, this is where the R&D research comes in. Rohan, thanks for coming in. Appreciate it. Going forward, re:Invent six months away. We're midpoint in the year. Halftime. What's the locker room pep talk to the partners/ What would you say out there what's coming?

Rohan Karmarkar

>> Yeah, I mean, see, even before re:Invent, there's New York Summit, there's a bunch of new things that we expect to come out New York Summit. So I think a lot of capabilities, the pace is not going to slow down. You're going to see a lot of innovation and coming into re:Invent, there'll be a lot of exciting opportunities for partners to kind of work with AWS.

>> We'll be in New York as well. Thanks for coming on.

Rohan Karmarkar

>> Thank you.

>> It's going to be fun. New York City Summits, everyone's in town, financial services, life sciences, insurance, all the verticals are there. And so that's one of my favorite summits, by the way, New York.

Rohan Karmarkar

>> Absolutely.

>> It's a good one. Thanks Rohan. Okay. I'm John Furrier here for the halftime report. Thanks for watching.