Clips
News
More from CDOIQ Symposium 2024

Nusrath Mohammed

Data Practice Leader

Tata Consultancy Services

AI data cleansing takes center stage in evolving data governance landscape

Anyone who has worked in the data industry for several decades can speak to its continued evolution. Today, the focus has shifted to artificial intelligence, emphasizing AI data cleansing and governance, as clean data is essential for effective AI implementation.That’s an emphasis that many have been pushing forward in this new era. Data must be cleansed for analytics to be more reliable, according to Nusrath Mohammed (pictured), data practice leader at Tata Consultancy Services.Tata Consultancy Services’ Nusrath Mohammed talks to theCUBE about AI data cleansing.“Now, because of AI, everybody wants to jump on that bandwagon

play_circle_outline Evolution of data from statistics to big data to AI

play_circle_outline Business case for cleansing data for gen AI

play_circle_outline Utilizing gen AI at all data levels for better insights

play_circle_outline Challenges of governing open data formats with multiple data platforms

play_circle_outline Need for traditional problem-solving skills alongside gen AI

Info
Transcript

Nusrath Mohammed | Tata Consultancy Services

Nusrath Mohammed

Data Practice Leader Tata Consultancy Services

Nusrath Mohammed, data practice leader at TCS, discussed the importance of data cleansing for analytics and AI. The business case has evolved with the rise of AI and gen AI, offering new opportunities for insights and improved data quality. Challenges of governing open data formats in the complex data landscape were also touched upon. Nusrath emphasized the need for a holistic approach that incorporates traditional tools alongside new technologies like gen AI or Purview. The potential for intelligent applications to help users make informed decisions based on... Read more

explore Keep Exploring

What is the importance of data cleansing in relation to implementing AI technology? add

What has been the impact of artificial intelligence on the need for organizations to cleanse their data, and how has the business case for data management evolved as a result? add

What is the importance of utilizing gen AI at all levels of data processing, including at the data inception stage? add

What innovative method of data governance was presented by AWS during a presentation on Monday? add

What factors must be considered in determining if a solution, such as Purview, will solve all problems, even if it involves gen AI technology? add

bolt Powered by CUBE AI

Nusrath Mohammed | Tata Consultancy Services

>> Hi everybody. Welcome back to CDOIQ 2024. We're here in Cambridge, Massachusetts at the Hyatt Regency overlooking the Charles River. The boats are out, they're rowing. I'm Dave Vellante. Paul Gillein is also here along with Sanjeev Mohan. We've been here two days of wall-to-wall coverage of CDOIQ, the premier Chief Data Officer conference. This is the 18th year of CDOIQ, the 8th year for theCUBE covering this event. I'm really excited to introduce you to Nusrath Mohammed, who's the data practice leader at TCS, Tata Consulting Services. Nusrath, welcome to theCUBE. Thanks so much for coming on. It's great to see you.

>> Thank you, Dave. Thank you very much. Nice seeing you. And thank you. I have wanted to meet you for the longest time. I've been following you since last 10 years. And you and John are my... The heroes, the Kevin Costner and Mel Gibson of the Data World.

>> Ah! Seriously. Come on. That's amazing to hear from-

>> It's lovely to meet you.

>> Thank you. Thank you so much.

>> You're making my dream come true. Thanks a lot.

>> Well, it's our pleasure, believe me. So tell us about your role at Tata and your background.

>> Yeah, definitely. So first about my background. I have over 20 years of experience in data and data world. First of all, started out as an IT, what do you call, consultant with the SAP. And then from the get-go itself, I started with data. And so that kind of got me into data world and loved the statistics. And so that kind of got me excited. And from there on worked with the Department of Defense and with large number of systems, bringing those systems into one platform. And I'm the data lead for those aspects. And that kind of kept me challenged, entertained, and going. And from there, what do you call, started working on building strategies. When we are migrating data, we are migrating from one system to another. And then what happens is we have to cleanse that data. And then now after we cleanse it, then how do we keep it clean? And that kind of answering that question kept me into, what do you call, let's bring the data governance, have a much more solid data strategy from start to end and ongoing and things of that sort. So now with TCS, that's what I help with the clients offering from start to end, from conceptualizing, from building a business case on why do we need data to be cleansed, and that is the biggest aspect. Cleansing is very tedious, long, lengthy work, so we have to build convincing business cases and bring from leadership to everywhere downwards, that this is important for your organization and your culture. And then from there, how do we keep it clean? So those are some of the things, and that's exciting for me.

>> So you've gone from practitioner to now helping other practitioners and you've seen quite a journey. I mean, it used to be SBSS and SaaS. A statistician or somebody who was really good with statistics, that was the extent of data. And then they all became data scientists. Some were true data scientists, some were just stat pros and the whole Hadoop movement and big data. But people really, I think, generally underestimated the exponential growth of data and what you could do with it. And then now, I mean, fast-forward, there's a very compressed version of history here, but then now you have AI coming in. And it's interesting to hear you talk about the business case for data cleansing because the bromide is if you don't have clean data, you don't have AI.

>> You don't have AI. And that's the saddest part. We have been propagating or we have been trying to bring that awareness to the organizations that you need your data cleansed for your analytics to be more trustable, reliable, things of that sort. But it was not hitting to the heart or to the core I guess. And now because AI, everybody wants to jump on that bandwagon, but then now we realize data is not cleansed and then we have to go backwards and start cleaning that first, and then, okay, now we can jump on that bandwagon.

>> So what's the business case look like before gen AI for convincing leaders to cleanse their data? And what's the business case look like after? Has it changed or is it the same?

>> I mean, see, gen AI, the business case for gen AI, I mean it's good it came out. It's a beautiful toy to play with. So before gen AI, the business case was still AI was there still. Machine learning is there. So that has been since the past five years, I have been catching fire. So because of those, you have to have your data. So realization of data to be more tidy, more pristine has been more in the forefront because of AI. So gen AI, yeah, last one year, gen AI came. But even before that, AI was there. Even before that, the hype of big data was there. But big data wasn't working because still your foundational data is not clean. Your foundational data is not ready for you. So now the hype of big data is gone, but then you do have data. You have lots, large amounts of data, and it is growing exponentially as we see. I mean, maybe whatever the data we are creating in one day has been created. So current one day data creation is almost equivalent of what we have created since last hundred years. So that is the exponential growth of data. Now how do we manage it and how do we keep it controlled? And that is the biggest question, biggest challenge in the organizations. So how do we make sense out of that? But then again, still your foundational data has to be cleansed. They have to be ready to support that.

>> So let me ask it differently. So I get the business case for having cleansed data for-

>> Sorry.

>> No, no, that's okay. For dashboards, right? You're going to have better dashboards, you're going to have better agreement, you're going to serve the business better, you have better analytics. And then things like Snowflake made that even simpler, lowered the denominator component of the business case. Or Databricks, if you wanted to really do some heavy machine learning, you could apply that. And having cleansed data for those use cases undergoing.

>> On going. Yeah,.

>> Now gen AI comes in and it's like, "Okay." This is interesting. You said it's a toy because this gets to the heart of what I'm trying to get to, which I know I'm not doing a great job of asking the question, but you'll help me.

>> No, no, no. .

>> Because it's a toy, it's like, "Okay, where's the real value?" And so that's what I'm kind of getting to. Are clients saying, "Okay, I got to clean my data so I can take advantage of gen AI," but then what do they do with it? If you look at what people are doing with gen AI, it's like I call them very chatty applications. They're summarizing text or maybe they're making some images there, some of that. Maybe they'd be doing chatbots, they're writing code, the code assist. Okay, nice, but it's kind of toys. And so how much of the business case can you actually drive with some of these new use cases? It sounds like it's still the same fundamental. It's get your data house in order and then there's so many things you can do in analytics and maybe leveraging gen AI, maybe supply chain, et cetera. Does that make sense? What are you seeing in terms of the value? Where's the value around cleansing the data? And has it changed, is really what I'm trying to get to.

>> Yeah, I mean, sorry, I didn't answer it properly earlier.

>> No, you did. It was just, I'm trying to get to the heart of it and it's sort of a fuzzy thing, right? Because-

>> Yeah, it is. It was a little bit challenging. How do I integrate gen AI? I mean, I see a lot of tools out there right now on Monday, how Purview was showcased and it's a beautiful tool. But then I'm asking the question, that tool can be utilized only at the top layer once we have the data, data lake has been built. So it's at the top layer. But what about at the layer of the data inception? So when we dig in a little bit more on how gen AI can be utilized, so the answer in that group came about was, you have to put in gen AI at all levels. And that clicked, and that is where we can utilize gen AI. So during the data inception, when your data is coming into your source systems, then apply gen AI there and see, detect and give us a report up there and say, "Okay, your addresses are not matching their accurate addresses." So we are already doing a Google search or whatever the API you want to use and "Here is your data." Or even when the user is actually creating that data, you give the help, "Hey, your address, you might have put in the building number incorrectly here. This could be the right building number." So it is already detecting that and helping you, aiding you-

>> Makes sense....

>> from there. And then when you're doing at a change level, you're doing it in interfaces level. So you're putting those controls that gen AI in that levels. And then now you bring it at the top level. And now you want to bring your insights, 140 tables in your data lake or thousand tables in your data lake. And now you're doing it. Yeah, you're going to have much more better outputs too. And also it would be a little bit lesser to correct your data at that level. So this is how I see gen AI helping and supporting what we are doing and even for the end results.

>> Interesting. So it's not necessarily driving the business case per se. It's supporting the overall activity, making it easier, natural language, et cetera. You brought up Purview, which is interesting. And I want to ask you, I got another kind of long-winded question that you're going to have to help me through.

>> Go ahead.

>> So Purview is Microsoft's catalog, or catalog I'll call it, but it's more than just the governance, it's security, role-based access control, but it's not meant to compete per se with catalogs like Polaris or Horizon or Unity. It sits on top of those. So in thinking about where we're at with governance, actually if I go back to Hadoop days, it's very complicated, hard to do. Spark comes along, makes it somewhat easier, but you still need a lot of really smart people. You've got the cloud. So you've got catalogs in the cloud, you've got now these open source catalogs. Take the example of Snowflake. People said, "Okay, we'll get you data house in order to put it all into Snowflake." That was Snowflakes, "Put all the eggs in the Snowflake basket." And that was easy to do, but it wasn't practical for your entire data estate. So now people are saying, "We want open table formats like Iceberg. And we want to bring any compute to the data. We don't want to just separate..." You're technical, I'm not an engineer.

>> That's okay.

>> But we can geek out for the audience if you'd like. "But rather than just separating compute from storage, which did a lot of great things, made it cloud infinite capacity, et cetera, we want to separate compute from data." So we can bring any engine, whether it's Trino or Spark or Snowflake, any engine. We want to own the data. So now you have all these new formats coming out with Polaris and Horizon and Unity and they're open source. And then the big question is, how do I govern them? That's what I ask every customer. Okay, great. Have you chosen one?" "Well, we want open table formats." "How are you going to govern them?" "We don't know. We're not really sure yet." Again, I told you it was a long-winded question. What are you seeing there? What's your perspective? I'm really interested in your perspective thinking about the history of how data and big data, now AI data, have evolved, especially in the context of governance, which is your specialty area and how customers are thinking about solving that problem of marrying open table formats and governing them?

>> So governing open data or open data formats is going to be a little bit challenging when it comes to companies, organization, private companies, even now what you call federated or public sector. So it's going to be challenging on that aspect. But what I have seen presentation on Monday by AWS, so they have given a pretty nice, it's like flying saucer methodology of data governance, innovate and do small and then experiment and then go back and then govern your data. So that was interesting and I really loved that out of the box thinking of governance. And I want to apply that and see how it would work.

>> So that's interesting because I see it as, it's like Republicans and Democrats. So I think maybe a bad analogy these days, but AWS has many, many different data platforms. Like 15. You got Aurora, you got Redshift, you have DynamoDB, et cetera, et cetera, et cetera. But they don't have a unified metadata sort of strategy. The technical metadata is in Glue. Maybe I'd be able to-

>> They use Glue for interfaces, but they have S3, they have the .

>> You have S3, okay.

>> Yeah, they have S3.

>> But the operational metadata, the business metadata might be in data zones. I'm kind of making this up, but you've got different catalogs distributed all over the place. They haven't unified the metadata yet, is my sense. But maybe with S3 they've got the answer there. Whereas then you look at Snowflake, they're saying, "Okay, we're going to open source Polaris, which is the technical metadata, but we're going to keep Horizon. You got to pay for that. That's not open source, but it has all the role-based access controls. We're going to do managed Iceberg tables." And then you have Databricks saying, "We are going to open source the Unity catalog." They pushed a button on stage. I don't know if you saw that, Matei, Matei Zaharia. It was Snowflake and Databricks-

>> Databricks....

>> are like Hatfields and McCoys. So in the Snowflake conference, the week before, they open source Polaris. The week after, Matei, on stage pushed a button, an open sourced Unity catalog, like, "Ha-ha! Gotcha." But then when you dig into it, you actually need the Spark execution engine to actually take advantage of that. So is it really open source? It's open source, but... Okay, so customers are confused, right? And it comes back to how are you going to govern all this stuff? And the likely answer is, I'm going to have a little AWS, I'm probably going to have some Databricks, because we know there's high overlap in those accounts. I'm going to have some Snowflake. I'm going to have to make some choices whether I go Polaris or Horizon. And then I got... Everybody has Microsoft. So throw some Purview in there. And you have this mess. This is where you guys come in, this is heaven for Tata. It's like, "Great, all this wonderful complexity," right?

>> We're a hyperscaler. .

>> Yeah. Right. And Google. We forget about Google. Can't forget about Google. So how do you help me as a customer solve this? Because that's what my environment's going to look like. We know this. It's going to be-

>> It's a complex problem, yeah.

>> It's fragmented. But that's the way the world is, right? That's the reality of what this look.

>> Yeah, and that is the complexity. And then when you go in some of these gen AI is going to fix that complexity. No, it won't.

>> Right.

>> I mean, you have to have a little bit more of a traditional way of solving problems. And then gen AI might facilitate in solving those problems. That's what I want to hear from our presenters are things of that sort. So that is where I come in with that pinch of salt and say, "Is Purview going to solve?" I don't want to be...

>> No, no you have to be agnostic to the technology. I understand, but this is-

>> Yeah. Is this solution going to solve all my problems? Because you may have embedded gen AI in there. No, it's not. I mean, you have to put in some of that machine learning skills and the data science and the whole nine yards of it after probably gen AI has given me some feedback. Maybe I still need to apply the traditional tools to get to actual solution. So those are some of the challenges or the talk which I'm not hearing and which I would like to hear. It's like, "This is not my silver bullet. This is at least aiding me, and then going through."

>> That's the gap right now. I mean, Oracle would disagree with this, but we see the point of control moving from the DBMS to the governance catalog. But the point of value is sort of leaping over the governance catalog because the governance catalog is getting commoditized because everybody's open sourcing it. So now you think about, "Okay," with this whole idea of co-pilots and agency and being able to have unified metadata and be able to take co-pilots and systems of agents to take multiple tool chains to solve really hard problems like supply chain and so forth. But this is more futuristic. But intelligent applications that are data apps, using data to actually create apps, that's very futuristic, but something that we think is coming.

>> But still, I mean, they have to learn what we are doing, right? So the co-pilots might help, but not right now.

>> Right.

>> It'll learn from me what I'm doing on a day-to-day basis. And then it'll give me much more intelligent advice at the end. But I don't think it is giving me intelligent advice at this moment.

>> No, but the bottom line is you have to have clean data before you can do any of that. Nusrath, thank you so much for coming on theCUBE. We're out of time.

>> Thank you.

>> We'll leave it there.

>> I loved to talk to you.

>> It's a good back and forth.

>> I met my Mel Gibson today.

>> Stop. No, don't get up. Don't get up. Don't get up.

>> Oh, okay.

>> All right. Keep it right there, buddy. We're back right after this short break. You're watching CDOIQ 2024 from the Hyatt Regency in Cambridge, Massachusetts. We'll be right back. You're watching theCUBE.

>> Thank you.