theCUBE + NYSE Wired: Data + AI: Turning Data Into Knowledge for Autonomous Systems | Shan Rizvi, Thumos Care | AI Luminaries with Neo4j

Clips
More from theCUBE + NYSE Wired: Data + AI: Turning Data Into Knowledge for Autonomous Systems

Shan Rizvi

Founder, Builder & Context Artist

MindGraph

play_circle_outline Ending Agent Amnesia: Knowledge Graphs and Ontologies for Persistent, Incremental AI Memory

play_circle_outline Graphs as Substrates for Memory, Traceability, and Ongoing Learning: Who Owns Intelligence — Organizations or Model Providers?

play_circle_outline Production considerations: latency, scalability, reliable graph databases

play_circle_outline Ontology-Driven Epistemic Structures for Trustworthy Decision-Making: Auditable Lineage and Structured Context to Minimize Hallucinations

play_circle_outline Unstructured data ingestion (chats, transcripts) into graphs for context

play_circle_outline Success criteria: agents improve with use; intelligence owned by organization

Info

Shan Rizvi, Thumos Care | AI Luminaries with Neo4j

Shan Rizvi

Founder, Builder & Context Artist MindGraph

search

John Furrier

>> Hello, I'm John Furrier here with theCUBE at our NYSE studio. We have our Palo Alto studio connecting Silicon Valley and Wall Street. We meet with the leaders in the industry, talk about all the latest trends that are making it happen. AI is super hot. AI agents, physical AI and robotics down the road coming super fast. This is our AI Luminaries series and program. We highlight the leaders making it happen. A conversation around moving things from pilot and experimentation into production, the challenges, the decisions, the architects and architecture making it all happen. Our guest, featured guest here is Shan Rizvi, founder, builder, and context architect at Thumos Care, among other ventures. Thanks for coming on this program. You're a luminary as of the self-described title. Thanks for coming on.

Shan Rizvi

>> Thank you, John. That's very kind of you.

John Furrier

>> You're also on the frontier too. one of the things we talk a lot about on theCUBE is the role of data, you see data lakes. We've seen that wave. It's now expanding into this agentic data-driven market where there's a lot of new factors that people are looking at— real-time reasoning, intelligence. It really is tuned perfectly for AI because without context and context windows, you really can't do much. So the word ontology has become mainstream. It's been around for a while. We're seeing companies like Palantir. Everyone's talking ontologies. This is what we've been obsessed with. You've been working on this. So I guess my first question is, where are we on the hype-to-reality cycle on ontologies and graphs? Because it seems to me that people are figuring out that you can blend in graphs to data, connect things, and the results are fantastic. What's your take?

Shan Rizvi

>> I'd say that it's certainly more reality than hype. the stock price trend for Palantir is perhaps some evidence of it. But even beyond that, based on first principles, I think graphs and ontology are here to stay, primarily because as the models, the base models get more intelligent, the context architecture or application layer has to be intelligent as well. Currently, every time you start working with an agent, it's like an amnesiac person has just woken up. They rethink everything from scratch, get some documents, and then reason over it. And then the next time you come, all that thinking is lost. You could have spent a whole conversation discussing a particular hypothesis, and all of that information is lost. It's not traceable. And, that's not how the human mind works. So I've been personally quite interested in thinking about how the human mind processes knowledge, builds reasoning over time incrementally, and how that process of thinking becomes traceable. And graphs, with the right ontology, are the obvious substrate to me.

John Furrier

>> Talk about the reality and how that— and why it's popular. Just last week we were in Paris for the RAISE Summit. Philip from Neo4j came on. He talked about context. We had Lin, the founder of Fireworks. They're crushing it. You're starting to see specialized intelligence. You're starting to see specialized models. You start to see the formation of kind of a connected data world. Why are graphs so popular right now? Why is the enterprise all abuzz about knowledge graphs. Why is it important? What should people know?

Shan Rizvi

>> So knowledge is by definition structural. When you make an argument, philosophers have known this for thousands of years, but if you think of an argument, it has certain components. There's a premise, and there's some reasoning applied to the premise, and then you have a conclusion. And depending on the quality of the premise and the reasoning, that conclusion may or may not be valid. So there's structured knowledge and there is a spectrum in knowledge as well. Not every statement qualifies as a fact, and not all facts are equal. So when it comes to thinking, when it comes to cognition, structure is quite inherent. It's quite necessary. So that's one reason. Another is that thinking and reasoning are also recursive by nature. If you look at how agents do retrieval currently, it's very one-shot. You make a query, you embed it, and then you find some chunks using semantic retrieval. But that's not quite how thinking works. The way thinking works is more like you retrieve some facts from your memory, then you reason over it, you test some hypothesis, you see whether you have what you want, then you go back and repeat that process a number of times till you have the answer that you were looking for. So recursion is the second point. And I think traceability, the need for transparency, is another point where if an agent makes a decision or even suggests a decision, you want to be able to attribute that to exactly what facts and what reasoning went into that decision. That's another thing, another area where, graphs with the right ontology solve a very critical problem. And finally, I'd say, people like Alex Karp from Palantir and Satya Nadella from Microsoft have been talking about the question of ownership of ongoing learning, right? So if enterprises are using AI models, all the expertise about their domain, the value of that is accruing with the model providers. Now, if it were to accrue with the organizations themselves. That's another place where graphs can be useful because if you're capturing the structure of knowledge, reasoning, decision-making, planning, and your world model in a graph that graph is growing every time you use it. It's not thinking from scratch every time. It's a bit like you're growing a tree of knowledge that just, gets bigger and better every time you use it. And then that graph can actually— is a really high-value dataset that you can use to fine-tune your own models and be model agnostic.

John Furrier

>> That's a great point. It's like I explained to a friend about training and inference. training is like going to school, and when you graduate, you've been trained. Now you infer, you learn and reason. And I like the angle about the brain because enterprises have never really had a system of record that could be thinking. They have their company brain as their moat. And I think Alex and many others are saying the data is the moat. Yeah, we've heard that. But the brain and the intellectual capital is the moat. So I want to ask you, because the memory comes up a lot, you do a lot of graph memory. You mentioned traceability, reasoning. Most enterprises have a search mindset versus, say, a graph mindset or a thinking mindset, because search is great. You put a query in, you get an answer. First answer out is fastest. But what you're saying is that might not be the right answer. It's like us saying what comes to the top of our head. It might not be the right answer. Reasoning is not about search. It's about the data and the steps it takes to get there. Put that— what's your reaction to that? And how do you explain why graphs are better than old school architecture around data, data search, retrieval? you have— you need a search discovery system. Graphs do that. But talk about the difference between just getting a search answer versus a thoughtful, reasoned answer, and why graphs?

Shan Rizvi

>> So, I completely agree with what you said. And to answer your question, when you talk about reasoning, the way to get to the right answer involves following relational paths. I was just talking to a startup that's building an AI intelligence for truck manufacturers doing maintenance, and they're focused on truck parts. Now, if you wanted to find all substitutes for a part that were manufactured by a company that is based in New York and also manufactures these other parts, That's the type of answer that search would struggle to answer, especially when you are dealing with a knowledge base with millions of parts and SKUs, right? That's a perfect example of why graphs are a better substrate for reasoning compared to search, because this type of traversal is quite critical to it. And as I mentioned earlier, recursion is another key component as well. So you might need to traverse certain paths in the graph, but then you might need to repeat that traversal process a number of times in different directions to find the best piece of information that you're looking for.

John Furrier

>> The agent world is all hyped up. I love agents. Who doesn't love to have an agent work while you're sleeping, doing tasks? All cool. Talk about the reliability piece and the production side of this, because what we've been seeing is graphs accelerate, especially with the AI infrastructure performance capability. Ontologies are now mainstream because AI infrastructure is so fast. But when you add graphs to it, what is the reliability production piece? What do people need to know about bringing graphs in, attaching them to data sources, and getting them up and running in a reliable way?

Shan Rizvi

>> Great question. I've both built prototypes that were not scalable, that used open-source graph database platforms that would struggle to scale beyond a million nodes. And then I've used Neo4j in production for Thumos Care, where we have a medical knowledge graph with connections between diseases and symptoms and drugs, as well as clinical guidelines. I'd say that the number one thing to keep in mind is latency. That's a trade-off that I come across, on a day-to-day basis, because yes, reasoning involves recursion, but then you also need to be flexible enough in your tool design that if you are looking for a piece of information, you get that as fast as possible. And you only get into those recursive loops when you're dealing with more complex reasoning. So that latency question is important to consider. And with graphs, scalability is quite critical to consider as well. You need to be able to, if you're dealing with, say, like those truck parts that we were talking about, right? If you're dealing with millions of parts and tens of millions of SKUs, you need a graph database platform that can handle that reliably at a reasonable latency.

John Furrier

>> Talk about trust and trustworthiness, because you mentioned traceability earlier. We've seen this before in all kinds of infrastructure game around observability, recovery, memory. But now you're applying it to this data layer. It's almost a data engineering, AI infrastructure layer. What needs to happen to make things trustworthy? Because we don't want hallucinations. If you have good context and reasoning set up, How does that piece fit in? What do graphs do there? Data lakes did a good job putting all the data in one spot. You tap into it, it's great. But now what happens when you add graphs? How do you manage and how do you advise and how do you think about trustworthiness, lineage, reporting, traceability? These all kind of to me, come to me and say, you actually have the artifacts, you actually reason and have a path, 'cause you can store that. What's the impact there? Why is graphs important for that?

Shan Rizvi

>> So, there are a couple of things here. For building trust, traceability and auditability is important. Now, we were talking about ontology earlier, when people think of ontology in the way that Palantir applies it, people in my experience are thinking of either an ontology to represent a particular business operation or relations between concepts in a particular domain. To solve the problem you're talking about, I believe that we need an ontology for cognition. So you have the ontology for your business operations and your domain, but you need an ontology for epistemic structure, that has components like claim and evidence and hypothesis and argument. but you also have an ontology for decision-making and planning. So then in a graph, every decision that an agent makes, can be linked to the epistemic structure of knowledge, that feeds it, and the domain objects, that it pertains to. And this is the type of structure in which you can have a lot of trust. models can still, of course, hallucinate, a certain part of that hallucination problem will be solved at the model layer. But honestly, state-of-the-art models are— the baseline level of intelligence is quite good, such that if you have a graph that has these cognitive primitives and this traceability, and you have the right tools to do such tracing and linking and qualification of decisions and reasoning and facts, you can be reasonably confident that a state-of-the-art model would minimize hallucinations because of just the clear structure of context that it's provided.

John Furrier

>> Yeah, I think that's an amazing point. When you look at the data market. I mentioned data lakes earlier. You got to connect to data. So, I love this idea of the human brain, the company brain. That's an advantage if tuned properly. You get all the benefits that you're talking about. I think the ROI is pretty clear. Faster decisions, smarter, that drives revenue, hopefully efficiency. There's traceability. You can measure it. You can report on it, track it. That's a quality recursive loop you can manage. I love that. But when you get into the company brain, there's a lot of data everywhere. It's as if the brain hasn't been connected. How do you look at the data ecosystem? How should practitioners think about this? Because graphs are a no-brainer in my opinion. Ontologies will be standard. There'll be many of them, but they have to connect. Agents will need to connect to the data. You got Databricks, you got Snowflake, you have a lot of different data sources, CRM systems. every company has a plethora of data that was not really designed to be the functioning real-time brain of the company. Yeah. Never mind a competitive advantage. So, this is a huge kind of IT meets data strategy with the graphs.

Shan Rizvi

>> Yeah.

John Furrier

>> Let's dive in. What is the state of the art? What's the best practice? What should people think about? Because this is a whole nother layer. This is not just a semantic layer, harmonization layer. This is a whole nother ballgame. Share your thoughts on this.

Shan Rizvi

>> So you're totally right about that. I've dealt with this question myself on multiple projects. You have information that is scattered in different sources and you have to decide what raw knowledge and reasoning should be in the graph. And what should you keep in the graph only at a structural level? So for example, if you have data lakes and conventional relational databases with a lot of information about your clients and your customers and your employees, in most cases it wouldn't make sense to copy all of that data into a graph even if there is some benefit in the structure. In that case, in the graph, you can store an ontological representation of your database tables so that the raw information itself that could be changing all the time, right, like sales are increasing, numbers are changing, to keep that in sync with the graph, may not be optimal. So you keep only the structure in the graph and that ontological structure in the graph allows you to then know what to query from those live data sources. Then there are unstructured sources like meeting transcripts, your chat platforms like Slack and Discord. Those I feel are more suited to incremental extraction, ingestion, and processing into a graph. So broadly, I guess what I'm saying is if you already have external structured data sources, you probably shouldn't copy that into a graph. But if you have unstructured sources, it makes more sense to extract and ingest them into a graph database. And in this context, the distinction between a graph database and a graph engine, I've also found to be helpful. Because the graph engine is more the part that models the ontology across these structured and unstructured data sources, whereas the graph database is the actual place where you store the graphs.

John Furrier

>> That's a great example because you think about Slack, a Slack thread, or even a chat thread, WhatsApp thread, anything that's unstructured has context. And we talked about memory before. Beyond retrieval augmented generation, which is finding what someone said in a chat, You're basically saying, let's make a graph out of that. Let's capture it. That's input to the overall ontology. It's a moment in time, but it becomes a timeless moment from a data standpoint versus thrown into a data lake and be like, okay, search. Did I get that right? is that the way to think about it?

Shan Rizvi

>> YeahI'd say that when you're processing unstructured data, it's better to process it directly into a graph rather than processing it into a data lake and then linking it to a graph. Again there are caveats there. I'm sure there are situations where that might make sense depending on the type of information, right? But the key point is that this architectural decision has to be made, like what type of data lives in a data lake and what type of knowledge lives in the graph database. And what is the graph engine layer that orchestrates and federates between these sources? For example, you're sitting at the NYSE, so if there's a data lake with the latest stock price and all the measurements like the P/E ratio that are changing all the time, that's in a data lake. An investment firm could have access to that data in their data lake, but then their own traders could be having conversations with potential clients and gauging demand for each of those companies. When you're dealing with an AI agent and you have a question about a particular stock ticker, you should get the context from a graph that seamlessly has both. So it's able to fuse the live information from the data lake with the information about those same entities that's processed from unstructured data like call transcripts and chat transcripts, etc.

John Furrier

>> think about the value of what you just said, because I think that is a huge point, because you have disparate context that's not linked, and it's absolutely complementary because you have stock price, which is just trading moments in time, snapshots of data, and the context of the conversation, in this case the brokers chatting or some other data source that's got contextual relevance. And so, we're using words like contextual relevance. Sounds like a search problem, but it kind of is, but it's not. How do you explain that to people and organizations? Because if they get this right, imagine the unlock in a brain model. How do you explain that to a team, a company really leaning into AI versus just rebuilding things from scratch? Throwing stuff here and throwing stuff against the wall. What's— how do you explain the value of that technically and just from a logic standpoint?

Shan Rizvi

>> HmmStarting from the logical part, I'd say that if you want to own a piece of the intelligence ecosystem, the piece that you own has to be seen as intelligence. Rather than data. And that abstraction is important because, as you were discussing earlier, intelligence is a lot more complicated. It has some structure, it has some specific components to it. So from a logical perspective, it makes sense to see that as intelligence and not as data and think about what is intelligence what is cognition like? How do you think? How does your brain make a particular decision? What type of knowledge does it retrieve? How does it process and reason over that knowledge? And what does that process look like? I have personally found that quite useful when I think about how I exercise intelligence. It becomes a lot easier to understand how these things should work. Then on the technical part, I'd say it depends a lot on what type of an organization you are and what type of intelligence you're building. But I would say that I'm personally quite bullish on graph databases. I think they are a critical piece of the puzzle, when it comes to traceability, auditability, or owning this layer of intelligence where you want yourself to have the type of data that allows you to own the ongoing learning that happens rather than just passing all your raw data to the model providers and letting them accrue all the value.

John Furrier

>> Yeah, I totally agree. I love this conversation. AI Luminaries here on theCUBE. That horizontal scale of data which provides an intelligence opportunity and you need low latency, right? You have real-time information. It's not like your brain has to ruminate over things and then pop an answer, go right to the right spot. That's what graphs do. Shan, great conversation. Congratulations on all the work you do and love what you do. Here on AI Luminaries. I think this is the key piece. We hear ontologies in the press, we hear people talking about it. It's been around for a while, but with all the advances in AI infrastructure and with graphs and the data work that's being done, it's a perfect opportunity. I guess my final question is, what does success look like for an organization? I'm sure people are beavering away and people have opinions. There's a little bit of dogma here, tech religion. What do the successful teams look like? Throw away the tech religion, clean sheet of paper, no preexisting database. What's the success formula for a practitioner who really wants to lean in and be a leader for their company to build that company brain?

Shan Rizvi

>> I'll start with the outcome that defines success. I think an organization would be successful in this endeavor if when you use your agents more, they get better and you own the piece that gets better. So you can switch models and the intelligence in your domain that's getting better, that's yours. You keep that. So that's how I would define success. As for how to get there, it would— the answer would basically be a summary of the conversation we have had so far, which is that you need to have a graph engine and database layer that is capable of storing the ontological knowledge, not just for your domain, but for cognition and decision-making that your agents are doing. And as long as you have those components, as long as you have an ontology for your domain and for knowledge and for cognition and for decision-making in your infrastructure, then you will be able to accrue that value. Then you will be able to have your models actually get better as they're used more frequently. And you would own that value that is accrued.

John Furrier

>> Yeah. I interviewed so many leaders over the years of theCUBE. It's our 17th year. I always ask, what does it take to be successful? This is like the CEO question. Well, be a good learner. Always be learning. And so, you hear words like graph mindset. People who have a graph mindset tend to do well. Always be learning. And what we're getting at here is that the AI has to keep learning. The context, the cognition. Great point. And graphs do that. So, Shan, thank you so much for coming on the program. Went a little longer than I thought, but great conversation as always.

Shan Rizvi

>> The pleasure is all mine, John. Thank you so much for inviting me, and it was a pleasure truly to talk to you about these things.

John Furrier

>> Yeah, it's super cool, and it makes a big difference. Companies can actually build that brain, and architects and practitioners can be the ones injecting that AI. You got to architect it. It's not a database, it's infrastructure. But that AI infrastructure will enable and unlock cognition, intelligence, thinking, real-time, all great stuff that's going to power the agents in the AI world. This is the AI Luminary Series. I'm John Furrier, the host. Thanks for watching.

Shan Rizvi, Thumos Care | AI Luminaries with Neo4j

search

John Furrier

>> Hello, I'm John Furrier here with theCUBE at our NYSE studio. We have our Palo Alto studio connecting Silicon Valley and Wall Street. We meet with the leaders in the industry, talk about all the latest trends that are making it happen. AI is super hot. AI agents, physical AI and robotics down the road coming super fast. This is our AI Luminaries series and program. We highlight the leaders making it happen. A conversation around moving things from pilot and experimentation into production, the challenges, the decisions, the architects and architecture making it all happen. Our guest, featured guest here is Shan Rizvi, founder, builder, and context architect at Thumos Care, among other ventures. Thanks for coming on this program. You're a luminary as of the self-described title. Thanks for coming on.

Shan Rizvi

>> Thank you, John. That's very kind of you.

John Furrier

>> You're also on the frontier too. one of the things we talk a lot about on theCUBE is the role of data, you see data lakes. We've seen that wave. It's now expanding into this agentic data-driven market where there's a lot of new factors that people are looking at— real-time reasoning, intelligence. It really is tuned perfectly for AI because without context and context windows, you really can't do much. So the word ontology has become mainstream. It's been around for a while. We're seeing companies like Palantir. Everyone's talking ontologies. This is what we've been obsessed with. You've been working on this. So I guess my first question is, where are we on the hype-to-reality cycle on ontologies and graphs? Because it seems to me that people are figuring out that you can blend in graphs to data, connect things, and the results are fantastic. What's your take?

Shan Rizvi

>> I'd say that it's certainly more reality than hype. the stock price trend for Palantir is perhaps some evidence of it. But even beyond that, based on first principles, I think graphs and ontology are here to stay, primarily because as the models, the base models get more intelligent, the context architecture or application layer has to be intelligent as well. Currently, every time you start working with an agent, it's like an amnesiac person has just woken up. They rethink everything from scratch, get some documents, and then reason over it. And then the next time you come, all that thinking is lost. You could have spent a whole conversation discussing a particular hypothesis, and all of that information is lost. It's not traceable. And, that's not how the human mind works. So I've been personally quite interested in thinking about how the human mind processes knowledge, builds reasoning over time incrementally, and how that process of thinking becomes traceable. And graphs, with the right ontology, are the obvious substrate to me.

John Furrier

>> Talk about the reality and how that— and why it's popular. Just last week we were in Paris for the RAISE Summit. Philip from Neo4j came on. He talked about context. We had Lin, the founder of Fireworks. They're crushing it. You're starting to see specialized intelligence. You're starting to see specialized models. You start to see the formation of kind of a connected data world. Why are graphs so popular right now? Why is the enterprise all abuzz about knowledge graphs. Why is it important? What should people know?

Shan Rizvi

>> So knowledge is by definition structural. When you make an argument, philosophers have known this for thousands of years, but if you think of an argument, it has certain components. There's a premise, and there's some reasoning applied to the premise, and then you have a conclusion. And depending on the quality of the premise and the reasoning, that conclusion may or may not be valid. So there's structured knowledge and there is a spectrum in knowledge as well. Not every statement qualifies as a fact, and not all facts are equal. So when it comes to thinking, when it comes to cognition, structure is quite inherent. It's quite necessary. So that's one reason. Another is that thinking and reasoning are also recursive by nature. If you look at how agents do retrieval currently, it's very one-shot. You make a query, you embed it, and then you find some chunks using semantic retrieval. But that's not quite how thinking works. The way thinking works is more like you retrieve some facts from your memory, then you reason over it, you test some hypothesis, you see whether you have what you want, then you go back and repeat that process a number of times till you have the answer that you were looking for. So recursion is the second point. And I think traceability, the need for transparency, is another point where if an agent makes a decision or even suggests a decision, you want to be able to attribute that to exactly what facts and what reasoning went into that decision. That's another thing, another area where, graphs with the right ontology solve a very critical problem. And finally, I'd say, people like Alex Karp from Palantir and Satya Nadella from Microsoft have been talking about the question of ownership of ongoing learning, right? So if enterprises are using AI models, all the expertise about their domain, the value of that is accruing with the model providers. Now, if it were to accrue with the organizations themselves. That's another place where graphs can be useful because if you're capturing the structure of knowledge, reasoning, decision-making, planning, and your world model in a graph that graph is growing every time you use it. It's not thinking from scratch every time. It's a bit like you're growing a tree of knowledge that just, gets bigger and better every time you use it. And then that graph can actually— is a really high-value dataset that you can use to fine-tune your own models and be model agnostic.

John Furrier

>> That's a great point. It's like I explained to a friend about training and inference. training is like going to school, and when you graduate, you've been trained. Now you infer, you learn and reason. And I like the angle about the brain because enterprises have never really had a system of record that could be thinking. They have their company brain as their moat. And I think Alex and many others are saying the data is the moat. Yeah, we've heard that. But the brain and the intellectual capital is the moat. So I want to ask you, because the memory comes up a lot, you do a lot of graph memory. You mentioned traceability, reasoning. Most enterprises have a search mindset versus, say, a graph mindset or a thinking mindset, because search is great. You put a query in, you get an answer. First answer out is fastest. But what you're saying is that might not be the right answer. It's like us saying what comes to the top of our head. It might not be the right answer. Reasoning is not about search. It's about the data and the steps it takes to get there. Put that— what's your reaction to that? And how do you explain why graphs are better than old school architecture around data, data search, retrieval? you have— you need a search discovery system. Graphs do that. But talk about the difference between just getting a search answer versus a thoughtful, reasoned answer, and why graphs?

Shan Rizvi

>> So, I completely agree with what you said. And to answer your question, when you talk about reasoning, the way to get to the right answer involves following relational paths. I was just talking to a startup that's building an AI intelligence for truck manufacturers doing maintenance, and they're focused on truck parts. Now, if you wanted to find all substitutes for a part that were manufactured by a company that is based in New York and also manufactures these other parts, That's the type of answer that search would struggle to answer, especially when you are dealing with a knowledge base with millions of parts and SKUs, right? That's a perfect example of why graphs are a better substrate for reasoning compared to search, because this type of traversal is quite critical to it. And as I mentioned earlier, recursion is another key component as well. So you might need to traverse certain paths in the graph, but then you might need to repeat that traversal process a number of times in different directions to find the best piece of information that you're looking for.

John Furrier

>> The agent world is all hyped up. I love agents. Who doesn't love to have an agent work while you're sleeping, doing tasks? All cool. Talk about the reliability piece and the production side of this, because what we've been seeing is graphs accelerate, especially with the AI infrastructure performance capability. Ontologies are now mainstream because AI infrastructure is so fast. But when you add graphs to it, what is the reliability production piece? What do people need to know about bringing graphs in, attaching them to data sources, and getting them up and running in a reliable way?

Shan Rizvi

>> Great question. I've both built prototypes that were not scalable, that used open-source graph database platforms that would struggle to scale beyond a million nodes. And then I've used Neo4j in production for Thumos Care, where we have a medical knowledge graph with connections between diseases and symptoms and drugs, as well as clinical guidelines. I'd say that the number one thing to keep in mind is latency. That's a trade-off that I come across, on a day-to-day basis, because yes, reasoning involves recursion, but then you also need to be flexible enough in your tool design that if you are looking for a piece of information, you get that as fast as possible. And you only get into those recursive loops when you're dealing with more complex reasoning. So that latency question is important to consider. And with graphs, scalability is quite critical to consider as well. You need to be able to, if you're dealing with, say, like those truck parts that we were talking about, right? If you're dealing with millions of parts and tens of millions of SKUs, you need a graph database platform that can handle that reliably at a reasonable latency.

John Furrier

>> Talk about trust and trustworthiness, because you mentioned traceability earlier. We've seen this before in all kinds of infrastructure game around observability, recovery, memory. But now you're applying it to this data layer. It's almost a data engineering, AI infrastructure layer. What needs to happen to make things trustworthy? Because we don't want hallucinations. If you have good context and reasoning set up, How does that piece fit in? What do graphs do there? Data lakes did a good job putting all the data in one spot. You tap into it, it's great. But now what happens when you add graphs? How do you manage and how do you advise and how do you think about trustworthiness, lineage, reporting, traceability? These all kind of to me, come to me and say, you actually have the artifacts, you actually reason and have a path, 'cause you can store that. What's the impact there? Why is graphs important for that?

Shan Rizvi

>> So, there are a couple of things here. For building trust, traceability and auditability is important. Now, we were talking about ontology earlier, when people think of ontology in the way that Palantir applies it, people in my experience are thinking of either an ontology to represent a particular business operation or relations between concepts in a particular domain. To solve the problem you're talking about, I believe that we need an ontology for cognition. So you have the ontology for your business operations and your domain, but you need an ontology for epistemic structure, that has components like claim and evidence and hypothesis and argument. but you also have an ontology for decision-making and planning. So then in a graph, every decision that an agent makes, can be linked to the epistemic structure of knowledge, that feeds it, and the domain objects, that it pertains to. And this is the type of structure in which you can have a lot of trust. models can still, of course, hallucinate, a certain part of that hallucination problem will be solved at the model layer. But honestly, state-of-the-art models are— the baseline level of intelligence is quite good, such that if you have a graph that has these cognitive primitives and this traceability, and you have the right tools to do such tracing and linking and qualification of decisions and reasoning and facts, you can be reasonably confident that a state-of-the-art model would minimize hallucinations because of just the clear structure of context that it's provided.

John Furrier

>> Yeah, I think that's an amazing point. When you look at the data market. I mentioned data lakes earlier. You got to connect to data. So, I love this idea of the human brain, the company brain. That's an advantage if tuned properly. You get all the benefits that you're talking about. I think the ROI is pretty clear. Faster decisions, smarter, that drives revenue, hopefully efficiency. There's traceability. You can measure it. You can report on it, track it. That's a quality recursive loop you can manage. I love that. But when you get into the company brain, there's a lot of data everywhere. It's as if the brain hasn't been connected. How do you look at the data ecosystem? How should practitioners think about this? Because graphs are a no-brainer in my opinion. Ontologies will be standard. There'll be many of them, but they have to connect. Agents will need to connect to the data. You got Databricks, you got Snowflake, you have a lot of different data sources, CRM systems. every company has a plethora of data that was not really designed to be the functioning real-time brain of the company. Yeah. Never mind a competitive advantage. So, this is a huge kind of IT meets data strategy with the graphs.

Shan Rizvi

>> Yeah.

John Furrier

>> Let's dive in. What is the state of the art? What's the best practice? What should people think about? Because this is a whole nother layer. This is not just a semantic layer, harmonization layer. This is a whole nother ballgame. Share your thoughts on this.

Shan Rizvi

>> So you're totally right about that. I've dealt with this question myself on multiple projects. You have information that is scattered in different sources and you have to decide what raw knowledge and reasoning should be in the graph. And what should you keep in the graph only at a structural level? So for example, if you have data lakes and conventional relational databases with a lot of information about your clients and your customers and your employees, in most cases it wouldn't make sense to copy all of that data into a graph even if there is some benefit in the structure. In that case, in the graph, you can store an ontological representation of your database tables so that the raw information itself that could be changing all the time, right, like sales are increasing, numbers are changing, to keep that in sync with the graph, may not be optimal. So you keep only the structure in the graph and that ontological structure in the graph allows you to then know what to query from those live data sources. Then there are unstructured sources like meeting transcripts, your chat platforms like Slack and Discord. Those I feel are more suited to incremental extraction, ingestion, and processing into a graph. So broadly, I guess what I'm saying is if you already have external structured data sources, you probably shouldn't copy that into a graph. But if you have unstructured sources, it makes more sense to extract and ingest them into a graph database. And in this context, the distinction between a graph database and a graph engine, I've also found to be helpful. Because the graph engine is more the part that models the ontology across these structured and unstructured data sources, whereas the graph database is the actual place where you store the graphs.

John Furrier

>> That's a great example because you think about Slack, a Slack thread, or even a chat thread, WhatsApp thread, anything that's unstructured has context. And we talked about memory before. Beyond retrieval augmented generation, which is finding what someone said in a chat, You're basically saying, let's make a graph out of that. Let's capture it. That's input to the overall ontology. It's a moment in time, but it becomes a timeless moment from a data standpoint versus thrown into a data lake and be like, okay, search. Did I get that right? is that the way to think about it?

Shan Rizvi

>> YeahI'd say that when you're processing unstructured data, it's better to process it directly into a graph rather than processing it into a data lake and then linking it to a graph. Again there are caveats there. I'm sure there are situations where that might make sense depending on the type of information, right? But the key point is that this architectural decision has to be made, like what type of data lives in a data lake and what type of knowledge lives in the graph database. And what is the graph engine layer that orchestrates and federates between these sources? For example, you're sitting at the NYSE, so if there's a data lake with the latest stock price and all the measurements like the P/E ratio that are changing all the time, that's in a data lake. An investment firm could have access to that data in their data lake, but then their own traders could be having conversations with potential clients and gauging demand for each of those companies. When you're dealing with an AI agent and you have a question about a particular stock ticker, you should get the context from a graph that seamlessly has both. So it's able to fuse the live information from the data lake with the information about those same entities that's processed from unstructured data like call transcripts and chat transcripts, etc.

John Furrier

>> think about the value of what you just said, because I think that is a huge point, because you have disparate context that's not linked, and it's absolutely complementary because you have stock price, which is just trading moments in time, snapshots of data, and the context of the conversation, in this case the brokers chatting or some other data source that's got contextual relevance. And so, we're using words like contextual relevance. Sounds like a search problem, but it kind of is, but it's not. How do you explain that to people and organizations? Because if they get this right, imagine the unlock in a brain model. How do you explain that to a team, a company really leaning into AI versus just rebuilding things from scratch? Throwing stuff here and throwing stuff against the wall. What's— how do you explain the value of that technically and just from a logic standpoint?

Shan Rizvi

>> HmmStarting from the logical part, I'd say that if you want to own a piece of the intelligence ecosystem, the piece that you own has to be seen as intelligence. Rather than data. And that abstraction is important because, as you were discussing earlier, intelligence is a lot more complicated. It has some structure, it has some specific components to it. So from a logical perspective, it makes sense to see that as intelligence and not as data and think about what is intelligence what is cognition like? How do you think? How does your brain make a particular decision? What type of knowledge does it retrieve? How does it process and reason over that knowledge? And what does that process look like? I have personally found that quite useful when I think about how I exercise intelligence. It becomes a lot easier to understand how these things should work. Then on the technical part, I'd say it depends a lot on what type of an organization you are and what type of intelligence you're building. But I would say that I'm personally quite bullish on graph databases. I think they are a critical piece of the puzzle, when it comes to traceability, auditability, or owning this layer of intelligence where you want yourself to have the type of data that allows you to own the ongoing learning that happens rather than just passing all your raw data to the model providers and letting them accrue all the value.

John Furrier

>> Yeah, I totally agree. I love this conversation. AI Luminaries here on theCUBE. That horizontal scale of data which provides an intelligence opportunity and you need low latency, right? You have real-time information. It's not like your brain has to ruminate over things and then pop an answer, go right to the right spot. That's what graphs do. Shan, great conversation. Congratulations on all the work you do and love what you do. Here on AI Luminaries. I think this is the key piece. We hear ontologies in the press, we hear people talking about it. It's been around for a while, but with all the advances in AI infrastructure and with graphs and the data work that's being done, it's a perfect opportunity. I guess my final question is, what does success look like for an organization? I'm sure people are beavering away and people have opinions. There's a little bit of dogma here, tech religion. What do the successful teams look like? Throw away the tech religion, clean sheet of paper, no preexisting database. What's the success formula for a practitioner who really wants to lean in and be a leader for their company to build that company brain?

Shan Rizvi

>> I'll start with the outcome that defines success. I think an organization would be successful in this endeavor if when you use your agents more, they get better and you own the piece that gets better. So you can switch models and the intelligence in your domain that's getting better, that's yours. You keep that. So that's how I would define success. As for how to get there, it would— the answer would basically be a summary of the conversation we have had so far, which is that you need to have a graph engine and database layer that is capable of storing the ontological knowledge, not just for your domain, but for cognition and decision-making that your agents are doing. And as long as you have those components, as long as you have an ontology for your domain and for knowledge and for cognition and for decision-making in your infrastructure, then you will be able to accrue that value. Then you will be able to have your models actually get better as they're used more frequently. And you would own that value that is accrued.

John Furrier

>> Yeah. I interviewed so many leaders over the years of theCUBE. It's our 17th year. I always ask, what does it take to be successful? This is like the CEO question. Well, be a good learner. Always be learning. And so, you hear words like graph mindset. People who have a graph mindset tend to do well. Always be learning. And what we're getting at here is that the AI has to keep learning. The context, the cognition. Great point. And graphs do that. So, Shan, thank you so much for coming on the program. Went a little longer than I thought, but great conversation as always.

Shan Rizvi

>> The pleasure is all mine, John. Thank you so much for inviting me, and it was a pleasure truly to talk to you about these things.

John Furrier

>> Yeah, it's super cool, and it makes a big difference. Companies can actually build that brain, and architects and practitioners can be the ones injecting that AI. You got to architect it. It's not a database, it's infrastructure. But that AI infrastructure will enable and unlock cognition, intelligence, thinking, real-time, all great stuff that's going to power the agents in the AI world. This is the AI Luminary Series. I'm John Furrier, the host. Thanks for watching.