Google Cloud Next '24 | Saurabh Mohanty & Scott Petry | PwC

Clips
News
More from Google Cloud Next '24

Saurabh Mohanty

AI Leader for Google Cloud Alliance

PwC

Scott Petry

Partner, Consulting Services, Cloud Engineering Practice Lead

PwC

Gen AI: Scaling smart solutions for enterprise-wide impact

Generative artificial intelligence has evolved beyond being just a promising tool; it has become an essential asset. With gen AI’s remarkable ability to comprehend, transform context and generate new content, it has already demonstrated significant value in enhancing business operations.Companies are successfully executing use cases with gen AI solutions but struggling to scale them into value-based solutions across their enterprise, according to Saurabh Mohanty (pictured, right), partner/principal at PricewaterhouseCoopers International Ltd

play_circle_outline Exploring the Future of Cloud Computing: Unveiling Collaborative Energy and AI’s Real Value at Google Cloud Next

play_circle_outline Overview of PwC's AI factory and its role in helping clients

play_circle_outline Mitigating Regulatory Risks: Developing a Governance Model for Ensuring Responsible AI Solutions

play_circle_outline Varying stages of AI adoption and value realization among customers

Info

Saurabh Mohanty & Scott Petry | PwC

Saurabh Mohanty

AI Leader for Google Cloud Alliance PwC

Scott Petry

Partner, Consulting Services, Cloud Engineering Practice Lead PwC

search

(upbeat music)

>> Good morning, Cloud Community, and welcome back to fabulous Las Vegas, Nevada. We're here in the start of day two of Google Cloud Next, "TheCUBE's" three day coverage this week. Absolutely fantastic. My name's Savannah Peterson, joined by analyst and superstar and first pitch thrower, Rob Strechay. Good morning, Rob. It's great to be here with you.

>> Well, at least you got my name right, you know, anyway, but no, it's great. I think what's been awesome, and I think we've talked about this a lot yesterday, is just the energy of the 30,000 people here.

>> Hundred percent.

>> But also that Google really has embraced its partners. And I think again, you know, we have another one with us right now. I think, again, it just shows that getting into AI, getting into data is really... It takes a village to get there, but there are people here to help you, and I think that's awesome.

>> And a lot of people are partnering together, it's why we've got fabulous folks from PwC, Saurabh and Scott, thank you so much for being here. How's the show going for you guys?

>> It's been fantastic. We get to talk about AI all day. (all laughing)

>> I was going to say, yeah-

>> It's my favorite topic, AI and engineering and how we can build stuff, and so it's fantastic. There's a ton of our clients are here, so we have a lot of relationships that we have the opportunity to build, and it's just a great energizing show to get to spend time, you know, with our partners and with our clients.

>> Yeah, and I think one of the big things that I always ask, because I think getting into AI, we have our own LLM, you know, but we're uber geeks, right? So we look at it. How do you help customers-

>> Speak for yourself, Rob. >> Well, yeah, well.

>> Kidding.

>> I wear the hat proud.

>> The nerd is probably right here.

>> How do you help really customers get into the first gen AI solution? What is like the gateway gen AI solution, and what are kind of some of the use cases that you see?

>> So I can talk about some of the things that we are seeing in the industry out there as it relates to generative AI solutions. I think companies have successfully been able to execute use cases, and those use cases are showing that there is an odd of possible in terms of what you can do with your generative AI, large language models. And, you know, Google has amazing models in their Vertex AI Model Garden, and companies are embedding that into multiple solutions. But what companies are struggling with right now is how do you take those use cases and how do you really scale them into value-based solutions that they can drive across their enterprise. So when you say first set of use cases, we are seeing it across the board, in marketing, in customer service, in product, in engineering, in operations, in procurement and finance, but it's the scaling that becomes challenging for these companies.

>> I think that, and also it's how do you find the actual value out of those use cases. So the implementation effort, the energy and time and money that you put into building out that solution, what are you getting out of it for real versus what is the hype that's not real and that doesn't actually create business value for your organization.

>> Oh, yeah. I mean, it's got to hit the P&L eventually. We all love a shiny MVP or show car when it comes to this type of technology and a lot of things. But if you're not actualizing that, if it's not actually improving the business, it's really just a distraction at that point and a cost center. So I can imagine you're really helping people navigate that. You've created an AI factory to help your customers. Can you tell me about that? Saurabh, we'll go to you first.

>> Yeah, so we try to do this on ourselves first saying we want to be client zero, and we said, here are our business models, here are our businesses, and this is what we want to do to reimagine the way we function in the future. And we said, for us to centralize all of this and have a seamless delivery model that's actually scaling these solutions, we need a COE, and we just call that COE AI factory, which basically consists of an overarching governance model with a governance team, and then delivery pods that are continuously delivering these solutions and deploying them at scale within our business.

>> That AI factory team has come up with a couple thousand use cases that we think are likely ones that ourselves and our clients would find value in. So they have a long backlog of opportunities to explore. We've invested in this AI factory for, I want to say it's like seven or eight years. So it's not all been gen AI, it's been AI since the beginning of that being a really important capability. And that's enabled us to be really quick to come to market with solutions for our clients because we had invested, we had explored, we had already been thinking about it, and like Saurabh said we're doing it to ourselves. So if you look at the other businesses that PwC has, like I represent engineering, he represents our data science and analytics team. But if you look at the businesses that we have in tax and accounting and auditing, there's a lot of process oriented workflows there that are ripe for automation and AI and generative AI solutions are core to that. So those are some of the big areas that we're applying it first for ourselves. For engineering, we're applying it to our software development lifecycle and trying to take advantage of every realistic use case in the SDLC. So that's another example. So the AI factories a critical enabler to make it possible for us to do that and then scale it out to ourselves.

>> Yeah, in fact, we have a partnership with a company called ETR that does tech spending intentions all the time quarterly. And what we've seen is that really to your point of being on the engineering side, we see that as kind of one of the gateway use cases for companies along with the marketing, along with some of the others. The one I want is tax, because like doing my taxes this year was a nightmare-

>> Oh, I'm here for it.

>> Yes. >> Amen.

>> So I look at this and go, it's really helping from a transformation perspective, especially within the workforce. What are you seeing around the transformations of the workforce and how it's happening and where it's going?

>> And that's really the crux of it for us being in the business that we are in, consulting our clients on how to operate their businesses more effectively. So the workforce transformation is the front end of whatever we end up getting to do with generative AI and engineering and building products for our clients. So to me, it goes back to that scaling problem though. Like workforce transformation is not a nice topic when you're automating jobs that are in a business process today. So the conversation about how you create value and how you structure your workforce and how you talk about that outcome of an automation or an improvement in the process is a huge conversation to figure out how you position that in an organization. Are you able to unlock people to generate more value? Are you really cutting staff? And especially with the kind of slow economy right now, it's a pretty sensitive topic when it comes to, you know, how do organizations best run their business, and how do we, as a society, take care of our people, take care of people. So it's a big topic for us, but I think it comes down to you have to be able to unlock value somehow if you're going to invest in that kind of a transformation. So where does it come from?

>> That's a really important point, and I think we love to talk about the shiny things when it comes to gen AI, but not the ethics and the governance. Saurabh, I want to ask you 'cause you're probably talking to a lot of different customers about this. How are you approaching conversations around ethical AI or even the messaging within orgs as they roll out different AI programs?

>> So we've identified nine different factors that they need to look at when they're looking at risk around AI. And that's just not limited to the bias or, you know, model accuracy, it also encompasses things like regulatory risk.

>> Huge right now. The whole regulatory market is so immature and underdeveloped, and everyone's kind of over here doing different stuff.

Yeah, anyway. >> Yeah, yeah, usage risk. Like how are people going to use it? Going back to Scott's point, right. You're doing your work in a certain way right now, tomorrow that's going to change. So how are you going to harness the usage and the power of the actual tool to do what you're supposed to do much better, faster, and in a more efficient way in the future. So there is a risk around adoption that you're dealing with when it comes to large enterprises, when people don't understand how to use it. There's a risk that's associated to, you know, the way that you are picking the models. Which LLM are you going to pick? Cost. Costs are going up.

Like there is risk- >> Exponentially with AI.

>> And so when we talk about scaling solutions, we actually put a lens of ROI on every solution saying, Hey, it's going to take you, you know, six months to build this, it's going to be 300 people that's going to use this in the organization. Your estimated costs are going to be X, and what's the business value? Y. Is it less than X, and why are you even going down that path? It doesn't even make sense.

>> You don't want to solve a $5 million problem

with a $6 million solution. >> Exactly. (all laughing)

>> I mean, again, that goes back to, you know, the ROI needs to be there. And I think to your point, and what use cases people will start with. I mean, do you really need a chatbot that's going to do X, Y, or Z, or is it really helping your customer success people service your customer better? And I think that's a lot of the conversations we're having this week. But one of the things you both have kind of touched on a little bit is the regulatory environment as well is changing. And I mean, you have the AI Act that just passed over in the EU, you have a number of different things coming down from a governance perspective that has to impact... PwC works globally, that has to impact how you look at deploying these solutions. Give us a little color on that from that perspective.

>> Yeah, absolutely. But especially as a regulated business, right, we're acutely aware of the regulatory risks associated with the things that we do, right. Like we advise our clients in many ways on how to manage their regulatory environment. It is incredibly challenging because the regulatory... The governments basically lag so much on what's actually happening, what's actually possible. So a certain part of responsible AI or ethical AI is to like think about what the right answer is and not wait for the government to make it a regulation. Like, what is a responsible way to behave with this kind of powerful technology? And how do you do that? You know, as a consultancy, we're advising our clients all the time on like, this is something that's responsible, this is something that would not be responsible. We have to take that position and, you know, basically take the high road ourselves and make sure that we're ahead of those regulatory outcomes whatever they end up being.

>> I mean, that's why we keep telling our clients to have a robust governance model in place that balances risk versus ROI. Because if you don't have that, you're not going to be able to prioritize what solutions you want to focus on, and you're not going to be able to generate real value out of it.

>> I think it's important that saying no is also not an answer. It's like, it's not an option to say, we're just going to block people from access to, you know, all of these large language models, and we're going to make sure nobody uses it by the firewall. That's not going to work, right? So you have to have a governance model that's actively engaged in enabling people to take advantage of the technology that's sitting there in front of them without doing things that are irresponsible.

>> I love that. I think that is one of the keys that I'll take away is, no is not an option. I mean, that to me is so true.

>> Yeah, it really is true. And how are your customers responding to this? Are they receptive? Are they sponges right now? Are people nervous?

>> So what we've observed is different customers are at different stages of evolution as it relates to adoption and actual value realization.

>> I can imagine, yeah. >> Yeah. There are some customers who are like basically in infancy where they're playing around with or experimenting with use cases. There are some who are really scaling it in a pretty big way and actually generating value. You know, our goal is to ensure that every customer that we work with is able to derive value out of these solutions in the best possible way. So we advise them on governance, we advise them on usage, we advise them on AI factory development, we advise them on, you know, business model reinvention, all of those components, leveraging the power of generative AI.

>> I think one key point there that we were talking about on the way over was a lot of boards have an OKR or KPI or whatever on generative AI adoption in their organization with no rhyme or reason to it. Like, adoption isn't really that important, but they realize it's an important enough topic that they need to move, they need to do something. So, you know, I think, it's good that there's a motive to go explore it. We have clients though, that are, like he said, at the infancy stage of it, and we have clients that are pretty advanced. And I've had a few clients that are taking, I'll call it a top down or outside in approach, and they're putting it on us. They're looking for us to cost less to perform the services because they're expecting us to have, you know, automated our own processes better, like make us faster, cost less.

>> Yeah, prove it. >> Give us a 10% discount because it's going to be less level of effort for this project.

>> Well, the other thing I'd like to add to that is, you know, AI has been there for years. Just because generative AI is now being used by people who don't have a data science background, it's just all the hype that got created. I think companies should think about governance, risk, responsibility, regardless of whether it's a generative AI solution or whether it's a machine learning solution or whether it's a deep learning solution.

>> It's really data, right, it's really data governance, that's the core of it.

>> I think actually you just hit on something that triggered something in my brain here around, how are you seeing organizations organize? Because to me, it's not just a data problem, it is an engineering problem. And a lot of times, all in vogue last year was, we're going to have a Chief Data Officer and they're going to solve our AI problem and they're going to come up with the use case. Are you seeing people kind of step back from that? Because it is not just a data problem, it's an engineering and a software issue as well. How are you seeing your clients kind of organize themselves?

>> You go first, I'll go next.

>> It's a little bit all over the place. I think that's a bit of the, where are they on the spectrum of adoption? I mean, I think in the concept that we're talking about with governance and having a properly organized governance, you really have to pick a place to centralize some of the decisions about how you're going to do things, and that gives you the right place to do data strategy and governance the right way, and I don't want to use the word controls, but put some emphasis on the right way to do it. Same thing with the engineering end of it, right? Like through our AI factory work, we've come up with a certain architecture that works really well for LLMs and building components on top of them that are usable for different use cases so that you get some scale with what you can deploy, like you get a couple of use cases going and you get a snowball effect. So getting that architecture right, getting the governance right, I think you have to kind of centralize it before you can democratize it, right. And I know everybody wants to democratize the data and democratize the tools, but you have to build that foundation before you can scale it. That's what I think I would like to see, I'm not sure I'm seeing that everywhere but.

>> I really agree with that though. I mean, John and I actually joke a lot on the show about democratization and what that actually means in practice, and there's so many things that would have to happen before this is actually democratized if we're talking about giving people access to stuff, and you do need a core group that can decide. I mean, these organizations you're working with are massive, so it's not like you can... If folks aren't all on board, there's just no way to do it. I'm curious, since you see so many different types of applications, and I realize there's probably some confidential stuff in here, so obviously share what you can. Are there any use cases that you're really excited about personally? Like when you talk to someone about it or this customer, you're just like, oh my gosh, I can't wait to see when that is real.

>> I think both of us are very excited about the application and software engineering from the inception of creating user stories based on requirements all the way to getting the test cases automated and then even all the way through CICD deployment. So I'm very bullish about that because I'm a software engineer by background, so I am biased.

>> You're allowed, I was asking your opinion, this was not a moment of data-driven decision making, this is- (all laughing)

>> Yeah, I'm in the same boat and I think we have started, like we said, to deploy these tools on ourselves in our software development life cycle. And there's a real measurable benefit, not only in time saved to do work, but the quality of the code that's produced and the quality of the delivered application is quite a bit noticeably higher. Like, without putting statistics on it on the show, but there's some real savings and real benefits there. And I'm as excited about the quality benefit as I am about the time to complete a task benefit, because the biggest problem we have in software engineering is quality at the end. So it's a huge win for us to get that benefit, so that's the one I'm excited about.

>> I will say, though, that I think there are three things that need to come together for all of these benefits to be realized. You need to understand the patterns that you're dealing with. And when I say patterns, I mean LLMs, vector databases, embeddings, and how you're really chunking and embedding your datasets. Now, when you have common patterns across your organization, you need to have reusability that you need to identify, and that's the job of a Chief Data Officer, we're talking about that. But with that, there are so many other personas. Personas is the second thing that I think is super important. All personas from business users to data officers, to legal counsel, finance, all of these personas need to come together to actually deploy a solution that works well within an enterprise. And then the last piece is controls, I think, which is synonymous to governance, which we were talking about. But yeah, I mean, we are both very, very excited about the applicability in software engineering.

>> I think there's a change management element in there too, 'cause the... I mean, engineers, we think we're good at engineering, but if somebody tells us we need to do it a different way, we're no better at changing than the next business user in their process. So we have seen places where half-hearted attempts to bring tooling into the process without really changing the process around it and changing the role that people expect to play in that process, basically it could have counterproductive situation, where it actually takes away or hurts the outcome that you're getting. So like, you don't want to deploy, for example, a backlog generator for your software engineering team without changing the role of the product owner in that system because now you've got a backlog that nobody's responsible for. Like, you're not going to get the right outcome, right? So you can't just throw the tool in there, you got to like go change the roles and the process to get it to work.

>> But people still matter.

>> Yeah, and we've actually created a solution on Google Cloud that we've been walking people through, it's actually in our booth as well.

>> We're going to have to go check it out then, yeah.

>> That's the most watched demo in the booth.

>> So what is it about?

>> It basically... I wouldn't say quantifies. It basically rolls out an entire life cycle of software development from requirements gathering into user story generation, into acceptance criteria documentation, into test case generation, code generation, test scripting automation, all the way to the end. And it also gives you an estimate in terms of how many people are required for building that specific user story and how are you going to allocate your capacity to do it in a pod structure. So it's a semi-automated solution, but I think it's pretty unique.

>> That is. Yeah, that's great. Well, we'll definitely going to have to come watch it, join everyone else. Last question for you as we wrap, when we have the opportunity to talk to you at this desk or at the next Google Cloud Next a year from now, what do you hope to be-

>> Is it going to be a year from now or seven months from now?

>> I'm hoping it's a year.

>> I honestly almost just said, or eight months from now or whenever. Who knows? Who knows? We're not going to be able to decide.

>> Maybe it'll shorten it to six months.

>> I wonder when the announcement's going to be. Anyway, we'll have to figure that out. What do you hope you can say that you can't say today? Scott, I'll start with you.

>> Oh, I hope we'll be able to say that we've figured out how to truly create business value out of this technology. And I kind of hope Google will be able to say more than speeds and feeds kind of stuff about the technology. Like, where's the business impact that really got created? I think that's going to be key, and I'm less worried about the co-pilots in the applications kind of area and more worried about the real hardcore business processes where we've built applications to enable people and now we're going to make those applications better and we're going to make the processes better and create value. I hope that's the story that we're talking about a year from now.

>> Here for it. What about you, Saurabh?

>> I'll be happy if I have a few clients that come and tell everyone that, look, PwC helped us reimagine our entire business model and reinvent it leveraging the power of generative AI and AI on Google Cloud. That's all there is to it. I mean, I'm very focused on a finite set of things that I can do.

>> Being focused on the customers. A very good place to- (indistinct)

>> 'Cause then you can share with us, then we'll get that use case, less secrets, more fun, I love it. Saurabh, Scott, thank you so much for being here. It's fantastic to have you on the show. Rob, always a pleasure, and thank all of you for tuning in wherever you might be on this beautiful day. We're here in Las Vegas, Nevada at Google Cloud Next. My name's Savannah Peterson. You're watching "TheCUBE", the leading source for enterprise tech news. (calm music)

Saurabh Mohanty & Scott Petry | PwC

search

(upbeat music)

>> Good morning, Cloud Community, and welcome back to fabulous Las Vegas, Nevada. We're here in the start of day two of Google Cloud Next, "TheCUBE's" three day coverage this week. Absolutely fantastic. My name's Savannah Peterson, joined by analyst and superstar and first pitch thrower, Rob Strechay. Good morning, Rob. It's great to be here with you.

>> Well, at least you got my name right, you know, anyway, but no, it's great. I think what's been awesome, and I think we've talked about this a lot yesterday, is just the energy of the 30,000 people here.

>> Hundred percent.

>> But also that Google really has embraced its partners. And I think again, you know, we have another one with us right now. I think, again, it just shows that getting into AI, getting into data is really... It takes a village to get there, but there are people here to help you, and I think that's awesome.

>> And a lot of people are partnering together, it's why we've got fabulous folks from PwC, Saurabh and Scott, thank you so much for being here. How's the show going for you guys?

>> It's been fantastic. We get to talk about AI all day. (all laughing)

>> I was going to say, yeah-

>> It's my favorite topic, AI and engineering and how we can build stuff, and so it's fantastic. There's a ton of our clients are here, so we have a lot of relationships that we have the opportunity to build, and it's just a great energizing show to get to spend time, you know, with our partners and with our clients.

>> Yeah, and I think one of the big things that I always ask, because I think getting into AI, we have our own LLM, you know, but we're uber geeks, right? So we look at it. How do you help customers-

>> Speak for yourself, Rob. >> Well, yeah, well.

>> Kidding.

>> I wear the hat proud.

>> The nerd is probably right here.

>> How do you help really customers get into the first gen AI solution? What is like the gateway gen AI solution, and what are kind of some of the use cases that you see?

>> So I can talk about some of the things that we are seeing in the industry out there as it relates to generative AI solutions. I think companies have successfully been able to execute use cases, and those use cases are showing that there is an odd of possible in terms of what you can do with your generative AI, large language models. And, you know, Google has amazing models in their Vertex AI Model Garden, and companies are embedding that into multiple solutions. But what companies are struggling with right now is how do you take those use cases and how do you really scale them into value-based solutions that they can drive across their enterprise. So when you say first set of use cases, we are seeing it across the board, in marketing, in customer service, in product, in engineering, in operations, in procurement and finance, but it's the scaling that becomes challenging for these companies.

>> I think that, and also it's how do you find the actual value out of those use cases. So the implementation effort, the energy and time and money that you put into building out that solution, what are you getting out of it for real versus what is the hype that's not real and that doesn't actually create business value for your organization.

>> Oh, yeah. I mean, it's got to hit the P&L eventually. We all love a shiny MVP or show car when it comes to this type of technology and a lot of things. But if you're not actualizing that, if it's not actually improving the business, it's really just a distraction at that point and a cost center. So I can imagine you're really helping people navigate that. You've created an AI factory to help your customers. Can you tell me about that? Saurabh, we'll go to you first.

>> Yeah, so we try to do this on ourselves first saying we want to be client zero, and we said, here are our business models, here are our businesses, and this is what we want to do to reimagine the way we function in the future. And we said, for us to centralize all of this and have a seamless delivery model that's actually scaling these solutions, we need a COE, and we just call that COE AI factory, which basically consists of an overarching governance model with a governance team, and then delivery pods that are continuously delivering these solutions and deploying them at scale within our business.

>> That AI factory team has come up with a couple thousand use cases that we think are likely ones that ourselves and our clients would find value in. So they have a long backlog of opportunities to explore. We've invested in this AI factory for, I want to say it's like seven or eight years. So it's not all been gen AI, it's been AI since the beginning of that being a really important capability. And that's enabled us to be really quick to come to market with solutions for our clients because we had invested, we had explored, we had already been thinking about it, and like Saurabh said we're doing it to ourselves. So if you look at the other businesses that PwC has, like I represent engineering, he represents our data science and analytics team. But if you look at the businesses that we have in tax and accounting and auditing, there's a lot of process oriented workflows there that are ripe for automation and AI and generative AI solutions are core to that. So those are some of the big areas that we're applying it first for ourselves. For engineering, we're applying it to our software development lifecycle and trying to take advantage of every realistic use case in the SDLC. So that's another example. So the AI factories a critical enabler to make it possible for us to do that and then scale it out to ourselves.

>> Yeah, in fact, we have a partnership with a company called ETR that does tech spending intentions all the time quarterly. And what we've seen is that really to your point of being on the engineering side, we see that as kind of one of the gateway use cases for companies along with the marketing, along with some of the others. The one I want is tax, because like doing my taxes this year was a nightmare-

>> Oh, I'm here for it.

>> Yes. >> Amen.

>> So I look at this and go, it's really helping from a transformation perspective, especially within the workforce. What are you seeing around the transformations of the workforce and how it's happening and where it's going?

>> And that's really the crux of it for us being in the business that we are in, consulting our clients on how to operate their businesses more effectively. So the workforce transformation is the front end of whatever we end up getting to do with generative AI and engineering and building products for our clients. So to me, it goes back to that scaling problem though. Like workforce transformation is not a nice topic when you're automating jobs that are in a business process today. So the conversation about how you create value and how you structure your workforce and how you talk about that outcome of an automation or an improvement in the process is a huge conversation to figure out how you position that in an organization. Are you able to unlock people to generate more value? Are you really cutting staff? And especially with the kind of slow economy right now, it's a pretty sensitive topic when it comes to, you know, how do organizations best run their business, and how do we, as a society, take care of our people, take care of people. So it's a big topic for us, but I think it comes down to you have to be able to unlock value somehow if you're going to invest in that kind of a transformation. So where does it come from?

>> That's a really important point, and I think we love to talk about the shiny things when it comes to gen AI, but not the ethics and the governance. Saurabh, I want to ask you 'cause you're probably talking to a lot of different customers about this. How are you approaching conversations around ethical AI or even the messaging within orgs as they roll out different AI programs?

>> So we've identified nine different factors that they need to look at when they're looking at risk around AI. And that's just not limited to the bias or, you know, model accuracy, it also encompasses things like regulatory risk.

>> Huge right now. The whole regulatory market is so immature and underdeveloped, and everyone's kind of over here doing different stuff.

Yeah, anyway. >> Yeah, yeah, usage risk. Like how are people going to use it? Going back to Scott's point, right. You're doing your work in a certain way right now, tomorrow that's going to change. So how are you going to harness the usage and the power of the actual tool to do what you're supposed to do much better, faster, and in a more efficient way in the future. So there is a risk around adoption that you're dealing with when it comes to large enterprises, when people don't understand how to use it. There's a risk that's associated to, you know, the way that you are picking the models. Which LLM are you going to pick? Cost. Costs are going up.

Like there is risk- >> Exponentially with AI.

>> And so when we talk about scaling solutions, we actually put a lens of ROI on every solution saying, Hey, it's going to take you, you know, six months to build this, it's going to be 300 people that's going to use this in the organization. Your estimated costs are going to be X, and what's the business value? Y. Is it less than X, and why are you even going down that path? It doesn't even make sense.

>> You don't want to solve a $5 million problem

with a $6 million solution. >> Exactly. (all laughing)

>> I mean, again, that goes back to, you know, the ROI needs to be there. And I think to your point, and what use cases people will start with. I mean, do you really need a chatbot that's going to do X, Y, or Z, or is it really helping your customer success people service your customer better? And I think that's a lot of the conversations we're having this week. But one of the things you both have kind of touched on a little bit is the regulatory environment as well is changing. And I mean, you have the AI Act that just passed over in the EU, you have a number of different things coming down from a governance perspective that has to impact... PwC works globally, that has to impact how you look at deploying these solutions. Give us a little color on that from that perspective.

>> Yeah, absolutely. But especially as a regulated business, right, we're acutely aware of the regulatory risks associated with the things that we do, right. Like we advise our clients in many ways on how to manage their regulatory environment. It is incredibly challenging because the regulatory... The governments basically lag so much on what's actually happening, what's actually possible. So a certain part of responsible AI or ethical AI is to like think about what the right answer is and not wait for the government to make it a regulation. Like, what is a responsible way to behave with this kind of powerful technology? And how do you do that? You know, as a consultancy, we're advising our clients all the time on like, this is something that's responsible, this is something that would not be responsible. We have to take that position and, you know, basically take the high road ourselves and make sure that we're ahead of those regulatory outcomes whatever they end up being.

>> I mean, that's why we keep telling our clients to have a robust governance model in place that balances risk versus ROI. Because if you don't have that, you're not going to be able to prioritize what solutions you want to focus on, and you're not going to be able to generate real value out of it.

>> I think it's important that saying no is also not an answer. It's like, it's not an option to say, we're just going to block people from access to, you know, all of these large language models, and we're going to make sure nobody uses it by the firewall. That's not going to work, right? So you have to have a governance model that's actively engaged in enabling people to take advantage of the technology that's sitting there in front of them without doing things that are irresponsible.

>> I love that. I think that is one of the keys that I'll take away is, no is not an option. I mean, that to me is so true.

>> Yeah, it really is true. And how are your customers responding to this? Are they receptive? Are they sponges right now? Are people nervous?

>> So what we've observed is different customers are at different stages of evolution as it relates to adoption and actual value realization.

>> I can imagine, yeah. >> Yeah. There are some customers who are like basically in infancy where they're playing around with or experimenting with use cases. There are some who are really scaling it in a pretty big way and actually generating value. You know, our goal is to ensure that every customer that we work with is able to derive value out of these solutions in the best possible way. So we advise them on governance, we advise them on usage, we advise them on AI factory development, we advise them on, you know, business model reinvention, all of those components, leveraging the power of generative AI.

>> I think one key point there that we were talking about on the way over was a lot of boards have an OKR or KPI or whatever on generative AI adoption in their organization with no rhyme or reason to it. Like, adoption isn't really that important, but they realize it's an important enough topic that they need to move, they need to do something. So, you know, I think, it's good that there's a motive to go explore it. We have clients though, that are, like he said, at the infancy stage of it, and we have clients that are pretty advanced. And I've had a few clients that are taking, I'll call it a top down or outside in approach, and they're putting it on us. They're looking for us to cost less to perform the services because they're expecting us to have, you know, automated our own processes better, like make us faster, cost less.

>> Yeah, prove it. >> Give us a 10% discount because it's going to be less level of effort for this project.

>> Well, the other thing I'd like to add to that is, you know, AI has been there for years. Just because generative AI is now being used by people who don't have a data science background, it's just all the hype that got created. I think companies should think about governance, risk, responsibility, regardless of whether it's a generative AI solution or whether it's a machine learning solution or whether it's a deep learning solution.

>> It's really data, right, it's really data governance, that's the core of it.

>> I think actually you just hit on something that triggered something in my brain here around, how are you seeing organizations organize? Because to me, it's not just a data problem, it is an engineering problem. And a lot of times, all in vogue last year was, we're going to have a Chief Data Officer and they're going to solve our AI problem and they're going to come up with the use case. Are you seeing people kind of step back from that? Because it is not just a data problem, it's an engineering and a software issue as well. How are you seeing your clients kind of organize themselves?

>> You go first, I'll go next.

>> It's a little bit all over the place. I think that's a bit of the, where are they on the spectrum of adoption? I mean, I think in the concept that we're talking about with governance and having a properly organized governance, you really have to pick a place to centralize some of the decisions about how you're going to do things, and that gives you the right place to do data strategy and governance the right way, and I don't want to use the word controls, but put some emphasis on the right way to do it. Same thing with the engineering end of it, right? Like through our AI factory work, we've come up with a certain architecture that works really well for LLMs and building components on top of them that are usable for different use cases so that you get some scale with what you can deploy, like you get a couple of use cases going and you get a snowball effect. So getting that architecture right, getting the governance right, I think you have to kind of centralize it before you can democratize it, right. And I know everybody wants to democratize the data and democratize the tools, but you have to build that foundation before you can scale it. That's what I think I would like to see, I'm not sure I'm seeing that everywhere but.

>> I really agree with that though. I mean, John and I actually joke a lot on the show about democratization and what that actually means in practice, and there's so many things that would have to happen before this is actually democratized if we're talking about giving people access to stuff, and you do need a core group that can decide. I mean, these organizations you're working with are massive, so it's not like you can... If folks aren't all on board, there's just no way to do it. I'm curious, since you see so many different types of applications, and I realize there's probably some confidential stuff in here, so obviously share what you can. Are there any use cases that you're really excited about personally? Like when you talk to someone about it or this customer, you're just like, oh my gosh, I can't wait to see when that is real.

>> I think both of us are very excited about the application and software engineering from the inception of creating user stories based on requirements all the way to getting the test cases automated and then even all the way through CICD deployment. So I'm very bullish about that because I'm a software engineer by background, so I am biased.

>> You're allowed, I was asking your opinion, this was not a moment of data-driven decision making, this is- (all laughing)

>> Yeah, I'm in the same boat and I think we have started, like we said, to deploy these tools on ourselves in our software development life cycle. And there's a real measurable benefit, not only in time saved to do work, but the quality of the code that's produced and the quality of the delivered application is quite a bit noticeably higher. Like, without putting statistics on it on the show, but there's some real savings and real benefits there. And I'm as excited about the quality benefit as I am about the time to complete a task benefit, because the biggest problem we have in software engineering is quality at the end. So it's a huge win for us to get that benefit, so that's the one I'm excited about.

>> I will say, though, that I think there are three things that need to come together for all of these benefits to be realized. You need to understand the patterns that you're dealing with. And when I say patterns, I mean LLMs, vector databases, embeddings, and how you're really chunking and embedding your datasets. Now, when you have common patterns across your organization, you need to have reusability that you need to identify, and that's the job of a Chief Data Officer, we're talking about that. But with that, there are so many other personas. Personas is the second thing that I think is super important. All personas from business users to data officers, to legal counsel, finance, all of these personas need to come together to actually deploy a solution that works well within an enterprise. And then the last piece is controls, I think, which is synonymous to governance, which we were talking about. But yeah, I mean, we are both very, very excited about the applicability in software engineering.

>> I think there's a change management element in there too, 'cause the... I mean, engineers, we think we're good at engineering, but if somebody tells us we need to do it a different way, we're no better at changing than the next business user in their process. So we have seen places where half-hearted attempts to bring tooling into the process without really changing the process around it and changing the role that people expect to play in that process, basically it could have counterproductive situation, where it actually takes away or hurts the outcome that you're getting. So like, you don't want to deploy, for example, a backlog generator for your software engineering team without changing the role of the product owner in that system because now you've got a backlog that nobody's responsible for. Like, you're not going to get the right outcome, right? So you can't just throw the tool in there, you got to like go change the roles and the process to get it to work.

>> But people still matter.

>> Yeah, and we've actually created a solution on Google Cloud that we've been walking people through, it's actually in our booth as well.

>> We're going to have to go check it out then, yeah.

>> That's the most watched demo in the booth.

>> So what is it about?

>> It basically... I wouldn't say quantifies. It basically rolls out an entire life cycle of software development from requirements gathering into user story generation, into acceptance criteria documentation, into test case generation, code generation, test scripting automation, all the way to the end. And it also gives you an estimate in terms of how many people are required for building that specific user story and how are you going to allocate your capacity to do it in a pod structure. So it's a semi-automated solution, but I think it's pretty unique.

>> That is. Yeah, that's great. Well, we'll definitely going to have to come watch it, join everyone else. Last question for you as we wrap, when we have the opportunity to talk to you at this desk or at the next Google Cloud Next a year from now, what do you hope to be-

>> Is it going to be a year from now or seven months from now?

>> I'm hoping it's a year.

>> I honestly almost just said, or eight months from now or whenever. Who knows? Who knows? We're not going to be able to decide.

>> Maybe it'll shorten it to six months.

>> I wonder when the announcement's going to be. Anyway, we'll have to figure that out. What do you hope you can say that you can't say today? Scott, I'll start with you.

>> Oh, I hope we'll be able to say that we've figured out how to truly create business value out of this technology. And I kind of hope Google will be able to say more than speeds and feeds kind of stuff about the technology. Like, where's the business impact that really got created? I think that's going to be key, and I'm less worried about the co-pilots in the applications kind of area and more worried about the real hardcore business processes where we've built applications to enable people and now we're going to make those applications better and we're going to make the processes better and create value. I hope that's the story that we're talking about a year from now.

>> Here for it. What about you, Saurabh?

>> I'll be happy if I have a few clients that come and tell everyone that, look, PwC helped us reimagine our entire business model and reinvent it leveraging the power of generative AI and AI on Google Cloud. That's all there is to it. I mean, I'm very focused on a finite set of things that I can do.

>> Being focused on the customers. A very good place to- (indistinct)

>> 'Cause then you can share with us, then we'll get that use case, less secrets, more fun, I love it. Saurabh, Scott, thank you so much for being here. It's fantastic to have you on the show. Rob, always a pleasure, and thank all of you for tuning in wherever you might be on this beautiful day. We're here in Las Vegas, Nevada at Google Cloud Next. My name's Savannah Peterson. You're watching "TheCUBE", the leading source for enterprise tech news. (calm music)