CDOIQ Symposium 2024 | Raj Joseph | DQLabs.ai

theCUBE.net

Clips
More from CDOIQ Symposium 2024

Raj Joseph

Founder & CEO

DQLabs.ai

play_circle_outline Raj Joseph founded DQLabs.ai to solve persistent data quality issues in organizations.

play_circle_outline DQLabs.ai offers a platform that balances business and engineering perspectives on data quality.

play_circle_outline The platform follows a four-step process: observe, measure, discover, remediate.

play_circle_outline DQLabs.ai focuses on the end-to-end data lifecycle and converging cataloging, quality, and observability.

play_circle_outline OEM relationships for cashflow

Info
Transcript

Raj Joseph | DQLabs.ai

Raj Joseph

Founder & CEO DQLabs.ai

Dave Vellante and Sanjeev Mohan are at the 18th Annual CDOIQ event in Cambridge, Massachusetts with Raj Joseph, founder and CEO of DQLabs.ai. Raj discusses his background in data and the founding of the company to address data quality issues. DQLabs offers a platform that balances business and engineering perspectives on data quality, providing capabilities to observe, measure, discover, and remediate data quality issues. Raj talks about the challenges of automating data quality and the importance of context and semantics in the process. He highlights the imp... Read more

explore Keep Exploring

What led to the creation of a solution for a long-standing data quality issue in the industry? add

What is the importance of data quality in the industry and how has it evolved over the past few decades? add

What factors determine whether an organization needs to focus on data operations or data quality, and how does the platform address these different needs? add

What are some key elements necessary to improve data quality in the end-to-end data lifecycle from a business outcome standpoint? add

What is DQLabs' approach to partnerships and raising money for their data intelligence platform? add

bolt Powered by CUBE AI

Raj Joseph | DQLabs.ai

search

>> Hi there, welcome back to Cambridge, Massachusetts. My name is Dave Vellante, here with my co-host, Sanjeev Mohan, Paul Gillin will also be here tomorrow. This is theCUBE's eighth year covering the 18th Annual CDOIQ event. Used to be the at the MIT, used to be associated with MIT. Now it's sort of broken off on its own and skyrocketing. Raj Joseph is here. He is the founder and CEO of DQLabs.ai. Raj, great to see you. Thanks for coming in.

Raj Joseph

>> Thanks for .

>> Fresh off the red-eye from LAX.

Raj Joseph

>> Fresh.

>> Love it.

Raj Joseph

>> I wouldn't quite call it as fresh, but I'm there hanging in.

>> Good man.

Raj Joseph

>> But thanks for having me and the pleasure meeting you again.

>> No napping on theCUBE, by the way.

Raj Joseph

>> Nope. Oh, yeah. With all the lights. No, not happening.

>> Okay. First question, why did you found the company?

Raj Joseph

>> I have a heavy background in data and so even from day one as an engineer, I started building data platforms. And all throughout my career, I was either being in data engineering or product management or dealing with data in terms of building. And one of the companies was Axiom. So I built the data platform. And going through all this process, everywhere I go, data quality was keeping on recurring. And even though it was recurring, there was not quite a clear solution that was unfolding. So we were kind of bits and pieces put together. So then I come like 2020, then we had this opportunity with a client, which was trying to do a solution around it and then found an opportunity and then saw like, okay, why don't we create a solution that the problem has not been solved for well over a couple of decades? So that's the passion and the reason began it.

>> Right. That makes sense. That's your why. We always talk about the why. So how? What's unique about what you guys do? What's your secret sauce? How do you approach it?

Raj Joseph

>> I think so if you have been watching the industry for the last 30 years or so, I think traditionally data quality has been a business problem. And it's been transforming in the last decade or two decades with the growth in data modern in engineering problem. And I think we are in a fascinating space where data is a fuel for AI and you have these two different perspectives of school of thoughts, and I feel like you need both of these perspectives have a balance of shift left and shift right? And that's where we position as a differentiation. So DQLabs is a SaaS platform, a platform that provides both this ability to see both from a business standpoint as well as from an engineering perspective standpoint, so.

>> How do you automate to the extent that you can data quality?

Raj Joseph

>> I mean, tough question. Because the data quality in itself, when you talk to different folks have different beliefs about quality. I mean, Sanjay, if they ask you, you would say a different version than what you may say, David. So I think the problem starts from there. So I don't try to see that we have to solve a hundred percentage other problem and it's a never-ending goal of sort. But what we do look into it is from a personal standpoint, if I'm a data engineer, what am I really trying to solve? If I'm a business user, if I'm a data steward, and try to see, cover the miles from that aspect of it. And there we have been very successful. Because there is always a percentage of piece that we can never solve and I don't think it's worth going for that. But if we can separate the percentage of the issues with the focus on 20, I think that's a great win. That's where we-

>> There's also, when we say automation, because there are two sites of the coin. One side is observing the data to infer if there is any quality. The data quality may not be apparent, but just applying some intelligence, that you can automate to some extent. But then there's the other side of the coin which is remediating and fixing it. That is very hard to automate.

Raj Joseph

>> True. And I think we have these four steps in the process. I think for me, a product is not a just product without a framework or a process thought through it. So in DQLabs we preach the four steps, which is observe, measure, discover, remediate. So observers, to your point, can we observe the whole ecosystem in a vast way, which is purely based on automation and alerting? But then the measure is the component where how do you know the data that you are using is fit for purpose? And that's where the question comes, not everyone sees the data is fed because today I'm building a simulation model, which is very different than a Gen AI , which is very different than a BA reporting analytics. So the fit for purpose changes, but giving the decentralized notion of measurement is critical, so that is a second step. But doing observe and measure mindlessly without any context then can . And that's where the semantics comes in, which is where the discover is putting a context, putting a meaning to the data and then applying some of this automation, makes it much more intelligent versus just going haywire everywhere.

>> Also, when you observe, it depends how deep of an introspection you're doing, right? Because I see that three layers of people observing, you can either observe a log and in the log you get more coarse-grained, you can observe the metadata and you get a little bit deeper or you can go to the cell level. But the moment you go to the cell level, you're actually asking SQL kind of-

Raj Joseph

>> Yeah, more like the content.

>> Yeah, the content.

Raj Joseph

>> The Data distribution.

>> And now you're putting more load, the cost can go up if it's snowflake, and you're asking these queries. So how does DQLab?

Raj Joseph

>> No, agreed. That's a great question. I think because if you go too deep, your cost goes up.

>> Correct.

Raj Joseph

>> If you go too shallow, then you are not providing the value. And so I think we start with more a shallow approach immediately out of the box, because the first thing you want to see any customer is a high bill price on day one. So we just take this crawl, walk, run approach, and then start off with a shallow mode out of the box. But immediately using the log information, we will be able to identify the criticality of the data assets.

>> I got it.

Raj Joseph

>> Right? There are some data which is more conceptions, more data, which is in the warehouse, which is very critical or important while there are some data which is just hanging in there. So doing a more deeper analysis for a table that is just hanging in there doesn't make any sense at all. So that's where your metadata comes in. Some of those information that you may have in a catalog also makes sense how much of this information is used and the consumption side becomes critical factor in decision. So these all put together becomes a glue in terms of the decision making process. But then it also makes how much of this automation can be really automated without the user feedback of sort? So that's where it considered bit great.

>> And Raj, are you saying that you could selectively go to that fine-grained detail?

Raj Joseph

>> Yep.

>> So you've got intelligence to allow you to do that, that you're inferring from metadata and log data?

Raj Joseph

>> Exactly.

>> But I want to come back to observe, measure, discover, and remediate.

Raj Joseph

>> Yep, remediate.

>> So are these features of your platform? Is observability a feature of your platform? Your ability to measure and discover. Did you build a semantic layer?

Raj Joseph

>> Yeah, we do have a semantic layer.

>> Okay.

Raj Joseph

>> And that is a differentiation between any of the other platforms in it. I strongly believe without the context, and this is a huge topic that's going to go even more traction with the AI, with the Gen AI and the LLMs going on, semantics is a bigger area of growth that I will think we will see. And I think because the amount of data that we are producing is enormous and . And without semantics layer, there is nothing that we can do meaningfully. So that's where we build the semantic layers. And then to answer your question on observe, measure, discover, remediate, that is just a framework and a process that we preach. But the platform has modules on observability as well as in quality, and the users can, or the organizations, depending upon the size of the organization. If I'm a small medium organization, very engineering focus, the data is very organic. I don't have any partner's data. I don't have multiple SAPs, CRMs, ERP systems, and it just like one or two traditional data warehouse, Snowflake, things like that. Then they don't need more of this business side measurement because data quality is not that critical. But what is more critical is data operations, data ops. And so there you can see the application of observe, discovery, remediate versus observe, measure, discovery, remediate. But in the other same, a large enterprise has both of these challenges. You have lots of data coming from different places you need to observe and monitor from a data operation standpoint. And then also the business, when you're creating different data products, you need to make able to measure and put a quality seal on top of it, then you may implement the four. So it's based on the size and different factors, but the platform gives all this modules and features.

>> So you'll work with observability platforms, for example, and ingest that data or?

Raj Joseph

>> No, we do have the platform-

>> Because that is a feature.

Raj Joseph

>> Yeah. We compete with observability platforms.

>> Oh, okay.

Raj Joseph

>> So we are an observability platform and a quality platform. So that convergence is the difference of sort.

>> Got it. And then the remediation piece is the holy grail?

Raj Joseph

>> Yep.

>> So that notion of a system of agency or Agentic AI, we're not there yet, but is that part of the roadmap or the vision?

Raj Joseph

>> No. So my notion of that is since we are monitoring, observing, measuring all this data, we know what is wrong, what went wrong and what should be the right value. So if you can give that information to a pipeline and trigger those workflows, then you are solving the end piece of that. So today what we do is we have specific triggers to specific DBT workflows and things like that, but we have not done across all the pipelines, but that is the goal of sort. So we are on the progression towards that end goal of doing an end-to-end data life cycle of fixing the problem. And the problem fixing is challenging because sometimes it's at the , sometimes it could be at the source level. So this is where it gets tricky, but I think our goal is to give that information to a dump-like pipeline, which could then become a smarter pipeline.

>> Because the data quality could be in the mainframe, so you cannot just go and change the mainframe. That's a very-

Raj Joseph

>> But if I provide this information to the data engineering team and then they could do what needs to be done.

>> Yeah, correct.

Raj Joseph

>> And the information is what makes it a little bit more powerful, just like in anything else.

>> Yes, correct.

Raj Joseph

>> The automation could be done in their tool stack by engineers or whatever they need to do.

>> Okay. But the AI can certainly, as part of the observation and discovery, can identify that and alert, I mean alert's maybe not the best word, but at least inform the data engineering team. And then if they like, presumably someday the AI can take action?

Raj Joseph

>> Yep. Yep.

>> Because you could have a customer in many different source systems and they are all silos of data. Now through the use of an LLM and doing the semantics, the distance search, you can find all these correlations and similarities across various different systems, extract that into a semantic layer and then do data quality on it. Is that?

Raj Joseph

>> Yeah, yeah, kind of. Because I think you cannot look into one specific table or multiple columns within a table. You need to have a overarching view across all your fields and all your data that's coming in, right? A file that does not drop in a data lake ADLS, ADLS may be the contributing factor for something else downstream. So there could be a lot of depending in the practice that could be. So having that overall semantic layer, observing, measuring, and then using that knowledge to make decisions definitely will help in the remediation aspect of it.

>> Although there is always a dark cloud inside the silver lining.

Raj Joseph

>> Dark cloud.

>> And the dark cloud is that when you have multiple, like LLMs can go and find these relationships, but when you've got these silos of data, they may have inconsistent information. Well guess what? That inconsistent information is now going to surface up in the semantic layer.

>> And it's got to be harmonized.

Raj Joseph

>> Yep.

>> Yes, it needs to be harmonized and somebody needs to clean it because LLM won't know which one is more recent. Therefore, for it, I've got two pieces of information that seem to be related, but then they're giving opposite values and which means the LLM is going to hallucinate.

>> So the catalog wars are actually a good thing for your business. So let's explore that a little bit. So what's happening is, we talked about this earlier, the point of control is shifting from the database management system to the catalog, but the point of value is leapfrogging the catalog because the catalog is getting commoditized. Everybody's open sourcing. You got Snowflake open sourcing Horizon, which is really the-

>> Polaris.

>> Sorry, Polaris, which is the technical metadata-

Raj Joseph

>> The Horizon is the .

>> The operational and business metadata is still inside and Horizon. Okay, so there's a hybrid model there.

>> And then Unity .

>> Databricks open-sourced on stage, Matei pushed a button and did Apache 2.0 with a Unity catalog. Of course Amazon has its version. Microsoft with Purview is trying to be a catalog of catalogs and add value atop of that. So how do you see all this and where do you see your firm adding value in this whole equation?

Raj Joseph

>> I think I don't see catalog as a competition to be honest. And then I see also the major technology players like AWS, Microsoft, Google, and then also Databricks, Snowflake, all the creating their own catalogs or even quality and observability. But I think the problem is I haven't seen one customer landscape where they just pick one technology vendor and they don't stick to it, even though it may be some cases beneficial, but I don't see that happening. So you always have a mix of technologies in a landscape that you go and then it becomes like Snowflake is very limiting. It cannot go beyond that. I don't see them pushing beyond their Snowflake landscape, and they would probably stay there. And that's where the value of a platform like us comes in because if you go SAP, then you're not going to be able to use Snowflake.

Raj Joseph

>> I think it's changing though.

>> Interesting. Yeah. If they don't extend beyond Snowflake, then somebody else is going to win the semantic layer war and then they could get commoditized.

Raj Joseph

>> I think it is changing though. I see these vendor specific catalogs now have connectors to SAP and Salesforce, and so they know that just being in their own domain will not cut it because then there'll be niche only for that product.

Raj Joseph

>> Yeah, I think it's just more the focus too, right? I think that's first point. The second point is Snowflake as a platform versus DQLabs as a company that is focused on quality and observability alone or anything towards improving the quality from an end to an lifecycle is very different. All I'm looking at is a glass of water while they're trying to look at across the ocean. So their benefits and features would be very shallow versus where we are doing very focused. So that's how I see it. But I think to answer your question, I think from a cataloging standpoint, I agree it's more of a reactive solution, even with active metadata and all that stuff, you're always taking a effort that needs to take a while to get the data in and all that stuff. And I think this is where a catalog plus quality and observability, all this will converge into one single platform of some sort. There is no other way in this market, because you need some of this information and catalog, you need some of this quality observability metrics to make the catalog even more valuable. And how do you leverage each other will become into this convergence. And that's what we started when we looked into it. We don't want to look quality or observability, we want to look the end-to-end data lifecycle from a business outcome standpoint, how can we improve the data quality? And that's not an easy problem to solve. Then you're going to start putting all these different capabilities, school of thoughts, technologies, and two other. So I think there will be more convergence to come.

>> Two things are very clear from our discussions with customers. One is open table formats are something that customers want. They want to go beyond separating compute from storage, they want to be able to bring any engine to their data in an open format. They want to control their own data, number one. Number two is, they don't know how to govern it and create data quality across all their data estates. It's very clear that to your point, Raj, that no one system is going to rule them all. So they're going to be just creating more stovepipes. So they need a solution like this. And so are you then partners with those platforms? Is that how ?

Raj Joseph

>> Yeah, we are partners with Snowflake, we are partners with Databricks, we are partners with SAP and all of those things. I think the more these vendors do more of this innovation actually helps more us because the more data that they provide, then we don't have to go create queries and create more conceptions. So it'll be there. So today, Snowflake usage schema gives us more with the Polaris and Horizon and Cortex and all of those things coming in. If they gave more of this information, then we get it right away. And so it makes our work less. But then also now we can really focus on the value chain of sort, how do we use this information? How do we combine it with the semantic layer? How do we make it more valuable? How do we put the data point from Snowflake and compare it to something that may be happening in SAP or the files that landing in ADLS? So these are the things that allows us to do it by doing so.

>> Where are you as a company? I'm just looking up on Crunchbase now.

Raj Joseph

>> As in location wise or ?

>> In terms of raise? How much money have you raised?

Raj Joseph

>> We are a pre-series A company. We have raised well over six to eight. But I think the focus is not about raising. I think today we are living in a generation of raising money and everyone talks about we are valued this and valued that, but I think we want to focus more on the customer use cases and the outcome. So for us, more customers, more wins, more value.

>> Well, the reason I ask is people want to know, how'd you start the company? How'd you fund the company?

Raj Joseph

>> I organically started.

>> So you funded it yourself?

Raj Joseph

>> I funded myself. So I also had a services company, and so I was able to take some cash from it and put it.

Raj Joseph

>> So the fallacy in Silicon Valley is the more money you raise, the higher is your profile. Actually you pay that money down the road. So I think what DQLabs is doing is very interesting because you have OEM relationships also.

Raj Joseph

>> Yeah. So we have Hitachi is taking DQLabs as Pentaho.

Raj Joseph

>> Pentaho.

>> Yeah. Yeah, we know Pentaho.

Raj Joseph

>> We also have another relation with Quest erwin. So there is multiple.

>> Oh, fantastic.

>> Quest erwin.

Raj Joseph

>> Yeah, Quest erwin. And so they use us a DQ for their data intelligence platform. And we do have significant other partnerships. So I think for us, I think the goal is not valuation or raising money. Of course it's critical and it's needed at different points of junction, but not something to be talked about it, but more about how do we solve the problems and how do we win this? So from the start, I've been very focused on that use cases and sort, and I think we both have talked about all these things.

>> That's great. It's very non-traditional way to start a company in California.

>> It's very non-traditional, yes, correct.

>> The OEM deals are fantastic, right? Because they can drive some cashflow if you can self-fund it and stay private as long as you can or stay self-funded as long as you can.

>> You'll be successful down the road.

Raj Joseph

>> Yeah. I think, the goal is not to stay private or this, at some point, it makes sense a lot to raise, but I think the problem we are solving is not an easy problem, to be honest. So I think raising a lot of cash and then trying to solve within a one-year period and putting a higher valuation actually creates more pressure and takes away the focus from the product and then puts into trying to do something that we cannot do within a shorter period of time so that we have been able to have the luxury of some sort in ways.

>> The failure rates, many people don't know this, but the failure rates of series A, B and C companies, you would think they decline the more mature you get, they don't.

Raj Joseph

>> They don't.

>> 70% of A, B and C companies fail.

Raj Joseph

>> And also-

>> Or have an exit .

Raj Joseph

>> Lately, the last five, 10 years, if you see the amount of companies which go series A from seed within a year is enormous, right?

>> Yes.

Raj Joseph

>> The whole A B, C has changed in terms of-

>> So the whole idea that getting funded is a badge of owner is not correct?

Raj Joseph

>> Yeah, I think so. I think it's a critical step that needs to achieve based on your cash flows and all, but not something like a proud movement to be like, because that's where your step zero starts and then you have more to do. And then you're also committing to some goals that you are committing based on timelines and things.

>> I think the point is that VCs are very good at making money for their LPs. They're not very good. They're terrible at creating successful companies. Their track record is 70% failure for A, B, and C. So it's a different perspective.

>> And by design, right? Because-

>> By design, exactly. I'm not saying that's just the wrong strategy if you're a VC or an LP, but if you're an entrepreneur, beware.

Raj Joseph

>> I also see lots of entrepreneurs get into this chasing, raising money every like, oh, we are not having any income revenue coming in, so let's go raise money. So it just becomes a addict of some sort. But then you losing away the focus of solving the problem. So for me, I always want to solve the problems. I want build a good product, a product that lasts and speaks volume. And so raising fund is critical, but not the focus.

>> Well, Raj, congratulations and best of luck to you. Really appreciate it. Thanks so much for coming on.

Raj Joseph

>> Thank you so much.

Raj Joseph

>> Thank you so much.

>> All right, for Sanjeev Mohan, Dave Vellante, this is theCUBE at MIT. Sorry, I almost said MIT. No longer MIT, CDOIQ from Cambridge, Massachusetts, right back this short break.