Snowflake Summit 2026 | Russell Spitzer & James Rowland-Jones, Snowflake

Clips
More from Snowflake Summit 2026

Russell Spitzer

Principal Software Engineer

Snowflake & Apache Polaris PPMC

James Rowland-Jones

Director, Product Management

Snowflake

play_circle_outline Interoperability Without Compromise: Snowflake, Apache Iceberg and REST Catalog APIs for Open Table Formats in the AI Era

play_circle_outline Access Data In Place with Snowflake: Eliminate Copies to Reduce Sync, Compliance, and Governance Risks

play_circle_outline Open standards prevent vendor lock-in and incompatible forks

play_circle_outline Polaris open-source metacatalog versus Horizon managed service

play_circle_outline Snowflake's Catalog Federation and Hub-and-Spoke Model for On‑Prem and Cloud Apache Iceberg

Info

Russell Spitzer & James Rowland-Jones, Snowflake

Russell Spitzer

Principal Software Engineer Snowflake & Apache Polaris PPMC

James Rowland-Jones

Director, Product Management Snowflake

search

Rebecca Knight

>> Hello, everyone. Welcome back to day two of theCUBE's live coverage of the Snowflake Summit here in San Francisco. I'm your host, Rebecca Knight, alongside Dave Vellante, co-founder and co-CEO of theCUBE. I would like to welcome our next guest to the show. We have Russell Spitzer, principal software engineer, Snowflake and Apache Polaris, PPMC. Welcome back, Russell.

>> Hi, it's great to be back again.

Rebecca Knight

>> Yes, wearing the uniform.

>> I am wearing the uniform.

Rebecca Knight

>> It must be said. Yes.

>> Yeah, I'm just saying.

Rebecca Knight

>> Next to you is James Roland Jones, director of product management at Snowflake. Welcome, JRJ.

>> Thank you very much. I'm in Snow White. So hi-

Rebecca Knight

>> Okay....

>> here we go. All right.

Rebecca Knight

>> Okay. All right, there you go. Okay.

>> Touche.

Rebecca Knight

>> I want to start with you, JRJ, because Snowflake is talking a lot this year about interoperability without compromise.

>> Yeah.

Rebecca Knight

>> Can you talk a little bit about what has fundamentally changed that makes open table formats and Apache Iceberg so critical in this AI era?

>> Yeah, absolutely. So interoperability without compromise is more of a mission statement for the team. What it actually means to deliver an interoperable ecosystem from the very foundations of your data, all the way up through to how you share products and data, both inter and intra organizationally. And what we look at in the team, and why Apache Iceberg is so important, is that it starts from an open governed project, an open source project, which Russell has been spearheading and working in that community for a long time, working with community members. And so you're already starting from an interoperable foundation, then you're building on top of that layers of interoperable access to the data, through things like the Iceberg Rest Catalog APIs. And so, as you come up through that stack, what you're trying to achieve is an ecosystem where any engine can both read and write to that data, whether that's a human or an agent being able to get direct access to that data. And in this AI powered era, that's what's so important, is being able to open that ecosystem up as much as humanly possible.

Dave Vellante

>> So Russ, if I go back to, I think it was Snowflake Summit 2022, Benoit Dageville asked for a show of hands in the audience. Many thousands of people, probably 10,000 people in the audience. "Who's ever heard of Apache Iceberg?" Very few hands went up. And then over the next couple of years, there was a lot of confusion, a lot of debate. There's competitive stuff going on. There was the Horizon, versus Polaris. And frankly, no offense, we spent a lot more time talking about this than we did about business value. That's largely, if not completely, gone away. Because Iceberg has matured, Polaris has matured, it's trusted now. It's much clearer. Can you take the audience through kind of where we came from, the journey, and where we are now?

>> I mean, it's a huge change, not just for the entire community, but for Snowflake itself. As an institution, Snowflake goes from that time when Benoit was asking those questions, to basically seeing there's a whole bunch of customers moving in this direction, or out there, with this data in a different format in a different system of governance. And they don't have any access there. So very slowly, we have changed the course of how the entire company is focused on interacting, and it's been great for both Snowflake and for all of those users, because now we have this whole section of people in the community who previously just couldn't even try any of the cool stuff that we're doing inside of Snowflake. And now, there's direct access to all of that, regardless of where you are thanks to these interoperable directions we're going in. And I think it surprised a lot of people I think, at Snowflake, at how successful this was. How willing people would be, if we took that first step and said, "We are embracing Iceberg. We are embracing this interoperability," to just jump on and start taking advantage of Snowflake. So for the company, obviously, a huge change. And the community, we really have just gotten to the point where everyone who's interacting with data says, "Iceberg is how we're going to do it. It means there's no friction. If I want to try out a new tool, if I want to try out a new library, a new engine." And of course now when we have agents and new AI things sprouting up all over the place, you have one foundational place where your data can be stored without any copies and you can access it from anywhere. So it's a win-win for pretty much everyone.

Dave Vellante

>> And I just want to follow up. You said it was slowly, and that's true, but I'll use an appropriate analogy. It's like the penguins on the iceberg, and then once the first one went, the rest flew. So it went, actually, relatively quickly once that you kind of broke through that area.

>> Yeah. We talk about this sometimes, in my old history as a biologist is, "a switch-like response." So you start very, very slow, and then once you hit an inflection point, suddenly everything shifts. Because it becomes obvious you're on the right direction, everyone is going in that way, and you're getting a lot of benefit from it. And then all your naysayers are stepping back, because now that's obviously where everyone is going, and it's time to get on board.

Rebecca Knight

>> So what does what Russell's describing unlock for the organization? I mean, obviously there's this great spirit of experimentation, and people are able to do things they hadn't been able to do before. But talk a little bit about the opportunity that that provides.

>> I think it's good to maybe think about a concrete example. We can now, as Snowflake, go to a customer who maybe has petabytes of data in a data lake. They have never used Snowflake before, but they can literally attach Snowflake to that data. And we call that "bringing Snowflake to the data," and you can literally show the value of what it means to use the Snowflake engines, and experiences and policies and governance, and all of these different capabilities over that data, literally that day. There is no load. You don't have to suddenly ingest your data. It used to be, like if we were to go back in, say back in time 10 years, the first thing you would do is bring your data, you'd load it into Snowflake, and then you'd be able to begin processing. Well, if you have petabytes of data, the last thing you want to do is move it, copy it, load it, any of those things. All of that has been eradicated by an open table format.

Rebecca Knight

>> I mean, if you think about this, I'm in a customer perspective. Even when I start trying that out, I'm in my mind thinking about, "How am I going to keep these in sync in the future? How am I going to make sure these two different copies are always accurate, that they're up-to-date?" But if there's one copy, that's not even a consideration. You can just start taking advantage of it with no worries. You can stop using Snowflake at any moment, you probably won't, because it's awesome. But you have that option. It's not a lock.

>> Yeah-

Dave Vellante

>> So... Go, please....

>> so we think of that as accessing the data in place. And really what you're seeing is, the central gravity is now like a cloud object storage bucket. And what you're really trying to avoid as an organization is siloing your data, and therefore introducing additional risks in that process, that people have known about but not really kind of internalized. Things like, if I copy the data and I load it into a proprietary system, well then, I have to now manage, say for example, a GDPR concern in two systems, or end systems. How do I know when I've actually solved that problem? I don't. I have to have very, very robust governance controls, that we see customers struggling with all the time and now those things, you have one logical copy of your data. So you are in a much, much stronger position, both for freshness, latency, and security and compliance concerns, with this approach.

Dave Vellante

>> So there's a strong openness theme, obviously, but I want to double click on that because a salient aspect of openness is interoperability. And so, Horizon now implements Polaris APIs. You've got sharing constructs for Iceberg-based data access, that's now inherent. So if we could talk a little bit about the importance of interoperability, some of the specifics of what the community has developed, and how Snowflake supports it.

>> I mean, it's one of our key values in both the Apache Iceberg community, which I'm a PMC member of, the Apache Polaris community that I'm a PMC member of, and we're also working on a new project called Open Semantic Interchange. Which, similarly, is about how do we get these things between people? And the reason that these projects have been successful and they spread so much, is that that group of people who are backing these projects realizes there's only a benefit to this if we remain interoperable. So every discussion is always focused on, "What are we doing? Are we making sure that everyone has consensus?" All the different vendors are looking at this and saying, "Yeah, we can work with that. This is the right next step," so that we never end up with a fork. Because that would be the worst possible scenario for all of the people who work on these projects, like me, very personally biased. Because if there's a fork, what's the point? If one group has their own version, and another group has their own specific version, now they're not interoperable and you might as well just be using a proprietary format. So that drive, that pressure, really makes sure that all of these communities focus completely on making sure that everyone is in agreement before we move forward.

Dave Vellante

>> Yeah. And I would say one of the genius things that the community did, when they moved from just supporting, say for example, Iceberg, the table format, into the API area. Was taking that sort of technology-agnostic approach of using the REST Catalog specification. And the reason for that is, that is agnostic to any implementation vendor choice. And I think, big kudos to the community for making that kind of brave decision, because what that has enabled is all of the kind of vendor implementations of those standards adhere to a technology-neutral based implementation. So I think that was very, very forward leaning from the community, across the board, like across the whole of the Iceberg PMC. So kudos to them, for that.

Dave Vellante

>> And Russell, to your point, monetizing forks, it's an anti-open source pattern, right? You got to find other ways.

Rebecca Knight

>> Yeah. I mean, and I think we're actually at a time now in the industry, where that just doesn't work. People see that you've done it and they're like, "I don't need it to be open source if you have the only supporters of this product. If you're the only ones maintaining and improving it, then why do I care that I can see the code?" That's not the real value. The value for me is that, I have that one copy of my data, everyone can read it. And I don't have to even think about, "Will my next tool, will the next big thing in AI work with my data?" Because if it's in Iceberg right now, it will.

Dave Vellante

>> And I think you're right. I think the industry realized this. When that happens, the committer base just consolidates down to one or two names that are sort of part of the ecosystem, and then it's... People, buyers don't walk, they run from that.

>> Yeah. And similarly, though, other things have been built on top. So the fact that when, with IRC, you also get locally scoped vended credentials, so that the community is not just thinking about the standards, but also the secure interoperability elements, and how that should play out across an ecosystem. Where, I think we fully recognize and are building, in a multi-catalog world. There is no enterprise out there that just says, "Oh, I'm going to have one catalog, or one vendor," or any of these things. And so we're working in that model, of a coexistence model, across these implementations.

Dave Vellante

>> I want to ask you about direction, because you mentioned OSI. I'm trying to connect the dots here. We kind of started out governing the technical metadata, bringing in the operational metadata. Now, with OSI, Cortex Sense, people are talking about context and ontologies, and so forth. Bringing in business process logic, and connecting to other parts. And this is the exciting part of where we're going, the business value on top. Can you give us a little, sort of directional angles, on where you see that going? How organizations are going to be able to bring in some of that tribal knowledge, if you will, and start to automate that?

Dave Vellante

>> I mean, I think that's actually the right perspective. We have this interchange of the data, we have this interchange of access to the data, but the things that we need now in this... I mean, this is hard to say if we'll need it in a year, but right now all of these agents to do really good work. If you want Koka to do the best work it can, if you want Cowork to work as good as it can, it needs the context beyond table names and identifiers. It actually needs to know the concepts that are tied to how your business actually operates. And we've seen, again, this happens all the time, we have a dozen competing variations on how this should actually be stored and exposed. And I mean, I think we're getting ahead of it here, hopefully, by saying, "We're going to work with other groups to say, 'This is how we'll define the exchange. We're going to say these are what the entities look like,'" so that now when your agent goes to a database, it's going to be able to pull in all of that context in a well-defined way, know exactly how to gather it all together. And your agent will be able to go off and run. Because we're kind of getting past that point where, the agent's going to write SQL, but it's going to write it with all of this other information as part of it. So I think that's the first future direction.

Dave Vellante

>> "And I'm talking to my data, too." A text to SQL. Yeah.

Dave Vellante

>> There's the thing, you're taking ownership, we call it this "agency over your data," right?

Dave Vellante

>> Yeah.

>> You're claiming control of the data, and you're using that. But it's really also, I think moving forward, we're seeing things like the importance of data products as the thing that kind of brings, all the things that Russell was talking about together, as a unit of sharing in a producer consumer model. Rather than thinking about data engineering as being like, "I am the creator of pipelines." Pipelines to do what? The future of this space kind of looks more like a data product owner shaping a data product for an organization, making that available to be shared both inter and intra organizationally, and then working back from say a model. Like I was speaking to one financial services company here at Snowflake Summit, and they were saying, "We start from the model and we work back to the data." And they're then using that as their pattern for sharing all the way across their organization. I think that's super exciting and very empowering. And I think for people that are questioning, in this agentic era, what it means for data engineers. I think actually, it's a really exciting time, because it actually does elevate them in the organization. But again, to your point, you have to take those steps.

>> I'd say, the analogy I would bring is, when we see LLMs in coding circumstances, it's pretty obvious. We've gotten to the point where your engineer does not have to write a for loop anymore. That's obviously not a part of a core competency. They're working at this higher level of describing the product, and describing what they want it to do, and maybe some technical aspects of how that works. And the same thing is happening in the data engineering space. We're probably going to be in a year or two away from anyone writing SQL. Instead, you're going to be writing, "These are the relationships between my different types of data, this is how they are structured, this is where you find this information, this is where you find that information." So it'll all just be at a higher level. I think we just keep going up the stack.

Rebecca Knight

>> That higher level is great for people who are seasoned, veteran data engineers, but what would you say to more entry level folks who are seeing the job market change before their eyes? And what kinds of skills, where should they invest their time, I guess I should say.

Dave Vellante

>> I mean, that's a really hard question. And I try to help a lot of folks entering the field right now. I do a lot of mentoring, and I try to talk to them about-

Rebecca Knight

>> Good deed....

>> what to do.

Rebecca Knight

>> Yeah.

>> And really, I try to say, "You need to position yourself really as this oracle in the system. You're no longer the person who's writing code." I mean, again, to give this analogy, there was a time where we didn't have good compilers, and everyone had to be an expert on, "Well, how do I unroll this loop to get the best performance out of this particular micro architecture?"

Dave Vellante

>> Squeeze it.

>> And then 10 years later, that was a nonsense skill. That was something that no one needed to do. But what you still needed to know is, "Well, what was the basic found fundamental thing that was happening in this processor?" So that when I architect at this higher level, I'm aware of the optimizations that are being made. And then, with LLMs, now I think we're just another step above that. "Are these encapsulations of the code correct? Are these patterns that are writing correct?" So I encourage people who are entering the field now to focus on still knowing those basics, but realize that your practice, the thing you should be doing on a day-to-day basis, is evaluating the products that are produced by these models and not by your own hand. So you want to be very familiar, but your job now is a reviewer, and a director. And it's not a typist.

Dave Vellante

>> That's good advice. And I'll follow up with that, because people say, "Oh, learn AI." Well, what does that even mean? What you just described is learn, really, how the fundamentals of the system work and then how to apply AI on top of it. And somebody yesterday, it might have been on theCUBE or in the evening event, said, "And then, figure out what the business needs, and work back from that."

Dave Vellante

>> Exactly.

Dave Vellante

>> "Because that's going to make your AI skills and your coding skills much more powerful."

>> Exactly. You're turning from, say for example, a pipeline which can often be considered a cost, and part of the cogs of running the business, into value creation. And you're trying to elevate people into that business, and thinking about the outcomes, and the value of what it is that they're trying to bring to bear in the organization. So yeah, that fully resonates with what I've seen.

Dave Vellante

>> I mean, the forward deployed engineers that you talk to will tell you that's the most important skill, is really understanding the business and being able to apply the problem.

>> It's always been true, right?

Dave Vellante

>> Yeah, right.

>> We've just taken some of the busy part out, and now that's your whole focus. You are only using your reviewing brain for the mechanical parts, but the business purpose, the design, that's still all mostly on you. Even if you ask agents to do designs for you, if you don't understand what you're actually trying to do, I mean, you're in a sea of hallucinations, right?

Dave Vellante

>> I got to ask you, as we close here, there's still some FUD in the market, about things like Horizon versus Polaris. You guys chose to have really distinct pass, and names, obviously work together. Help us understand the positioning, how you communicate to customers, where I should use each, how they relate to each other.

>> I mean, for me, so I'm the open source guy. I obviously love loading up all my own code, and running my own everything. So there's a bunch of customers who are like me, who are in those ecosystems where they're like, "I don't want to have anything handed over to anybody. So I'm going to run Polaris on my own, going to do my own hosting, I'm going to do this." I mean, maybe they even rent their own servers somewhere, and they're not using cloud service providers. So there is a whole class of folks over there who just want complete control, and they want to be able to dip their toes in Snowflake whenever they want, and those are Polaris users. And then in my mind, there's a shift over. If you're already a Snowflake customer, you already have all your accounts, your ID, everything is set up within the Snowflake ecosystem. We just open the door for all Iceberg usage to you through Horizon. So if you're already comfortable within the Snowflake ecosystem, you don't even have to think about this as a choice. You already have the option right there, and you know that if you ever do want to move, that's also completely available because of how these APIs are structured. I don't know if JRJ has a better answer.

Dave Vellante

>> I would say, the other thing, is that Horizon interoperability is based on Apache Polaris. So it is an open core that is powering your ecosystem. So that also means that it is not like two separate products. It is, you're graduating, as in Russell's example, you're graduating from a self-managed environment to one that is fully managed for you, but on the foundation that is Apache Polaris, so we develop in Polaris and then graduate that into Horizon. And that is a different approach to many of the other vendors that are out there in this space. That you actually have an open core that is present in your managed service. Now of course, on top of that, we then build on all of the other capabilities. Because for a true enterprise kind of metadata catalog, you need to be able to provide all of the security, you need to be able to provide your table maintenance and optimization, and all the governance components, as well as the metastore itself. So I think of it as almost that Polaris is your core, and then you have the value-add that Snowflake is providing on the top, but you have the assurance... Again, not that we ever want this to happen, but if you did decide you wanted to move away from Snowflake-

Dave Vellante

>> You can....

>> you know that you have a foundation that is the same.

>> I mean, I think it shows an amazing amount of trust, that we can open the store. And we know more people are going to come in than are going to go out.

Dave Vellante

>> Because I could choose to, the first group that, I could choose to build that sort of Horizon-like capability on top. If I want to do that. And I would say, by my experience in market analysis that I've done, it's maybe about 15% of the market that will actually choose to do that. They'll maybe regret it, we'll see. But that option is there, and the other point is when you say, "We're not locking you in," you can say, "Here's why." And that has teeth.

Rebecca Knight

>> Yeah. And also, what you're getting to, is also a hub and spoke model.

Dave Vellante

>> Yeah.

>> Because obviously Snowflake is available in all three major public clouds, but there is Iceberg data we know there's petabytes, exabytes of it, that lives in places like on-prem. But with our catalog federation capabilities and everything else, if you are using Polaris to power your, say, on-premises data lake, you can connect that into Snowflake, into the cloud, when you want to be able to share and enable the discovery of that data.

Dave Vellante

>> I'm so glad you brought this up. But go ahead, finish the thought, please.

Rebecca Knight

>> No, no, no. That's fine.

Dave Vellante

>> Because there is a lot of action going on, on-prem. I mean, you could see it in companies like Dell, are obviously resurging, because people don't necessarily want to move all their data into the cloud. They're bringing a lot of the intelligence in. And people say, mistakenly, "Well, Snowflake can't participate in that." But actually, based on what you just described, JRJ, they can-

Rebecca Knight

>> Absolutely....

Dave Vellante

>> actively.

>> Yeah. And we think of it as almost, rather than thinking about the world as, say, data integration. What you're doing is you're connecting to all of your data. And as Benoit was saying in the keynote the other day, the vision of Snowflake was always to be able to work across all data, not just data in Snowflake or tables, it is across all your data. And so as we move forward, it won't just be Iceberg on open kind of table structured data, I'm super excited for all the things the community is cooking up around enabling it for AI use cases and multimodal use cases. So Iceberg is not just for structured data, it is for every type of data.

Dave Vellante

>> And that was always the vision of Iceberg, is bring any compute to any data, and may the best engine win. And I would presume that Snowflake is comfortable competing at that level.

Dave Vellante

>> You're right, we feel very comfortable about that.

Rebecca Knight

>> Yes, indeed. Well, this is a great note to end on. Russell and JRJ, thank you both so much for coming on the show-

Dave Vellante

>> Thanks for having us.

>> Thanks, you guys....

Rebecca Knight

>> and our whole conversation.

>> Thank you.

>> See you next year.

Rebecca Knight

>> Indeed, we will. Rebecca Knight for Dave Vellante. Stay tuned for more of theCUBE's live coverage of the Snowflake Summit, just after a quick break. You're watching theCUBE, the leader in enterprise tech news and analysis.

Russell Spitzer & James Rowland-Jones, Snowflake

search

Rebecca Knight

>> Hello, everyone. Welcome back to day two of theCUBE's live coverage of the Snowflake Summit here in San Francisco. I'm your host, Rebecca Knight, alongside Dave Vellante, co-founder and co-CEO of theCUBE. I would like to welcome our next guest to the show. We have Russell Spitzer, principal software engineer, Snowflake and Apache Polaris, PPMC. Welcome back, Russell.

>> Hi, it's great to be back again.

Rebecca Knight

>> Yes, wearing the uniform.

>> I am wearing the uniform.

Rebecca Knight

>> It must be said. Yes.

>> Yeah, I'm just saying.

Rebecca Knight

>> Next to you is James Roland Jones, director of product management at Snowflake. Welcome, JRJ.

>> Thank you very much. I'm in Snow White. So hi-

Rebecca Knight

>> Okay....

>> here we go. All right.

Rebecca Knight

>> Okay. All right, there you go. Okay.

>> Touche.

Rebecca Knight

>> I want to start with you, JRJ, because Snowflake is talking a lot this year about interoperability without compromise.

>> Yeah.

Rebecca Knight

>> Can you talk a little bit about what has fundamentally changed that makes open table formats and Apache Iceberg so critical in this AI era?

>> Yeah, absolutely. So interoperability without compromise is more of a mission statement for the team. What it actually means to deliver an interoperable ecosystem from the very foundations of your data, all the way up through to how you share products and data, both inter and intra organizationally. And what we look at in the team, and why Apache Iceberg is so important, is that it starts from an open governed project, an open source project, which Russell has been spearheading and working in that community for a long time, working with community members. And so you're already starting from an interoperable foundation, then you're building on top of that layers of interoperable access to the data, through things like the Iceberg Rest Catalog APIs. And so, as you come up through that stack, what you're trying to achieve is an ecosystem where any engine can both read and write to that data, whether that's a human or an agent being able to get direct access to that data. And in this AI powered era, that's what's so important, is being able to open that ecosystem up as much as humanly possible.

Dave Vellante

>> So Russ, if I go back to, I think it was Snowflake Summit 2022, Benoit Dageville asked for a show of hands in the audience. Many thousands of people, probably 10,000 people in the audience. "Who's ever heard of Apache Iceberg?" Very few hands went up. And then over the next couple of years, there was a lot of confusion, a lot of debate. There's competitive stuff going on. There was the Horizon, versus Polaris. And frankly, no offense, we spent a lot more time talking about this than we did about business value. That's largely, if not completely, gone away. Because Iceberg has matured, Polaris has matured, it's trusted now. It's much clearer. Can you take the audience through kind of where we came from, the journey, and where we are now?

>> I mean, it's a huge change, not just for the entire community, but for Snowflake itself. As an institution, Snowflake goes from that time when Benoit was asking those questions, to basically seeing there's a whole bunch of customers moving in this direction, or out there, with this data in a different format in a different system of governance. And they don't have any access there. So very slowly, we have changed the course of how the entire company is focused on interacting, and it's been great for both Snowflake and for all of those users, because now we have this whole section of people in the community who previously just couldn't even try any of the cool stuff that we're doing inside of Snowflake. And now, there's direct access to all of that, regardless of where you are thanks to these interoperable directions we're going in. And I think it surprised a lot of people I think, at Snowflake, at how successful this was. How willing people would be, if we took that first step and said, "We are embracing Iceberg. We are embracing this interoperability," to just jump on and start taking advantage of Snowflake. So for the company, obviously, a huge change. And the community, we really have just gotten to the point where everyone who's interacting with data says, "Iceberg is how we're going to do it. It means there's no friction. If I want to try out a new tool, if I want to try out a new library, a new engine." And of course now when we have agents and new AI things sprouting up all over the place, you have one foundational place where your data can be stored without any copies and you can access it from anywhere. So it's a win-win for pretty much everyone.

Dave Vellante

>> And I just want to follow up. You said it was slowly, and that's true, but I'll use an appropriate analogy. It's like the penguins on the iceberg, and then once the first one went, the rest flew. So it went, actually, relatively quickly once that you kind of broke through that area.

>> Yeah. We talk about this sometimes, in my old history as a biologist is, "a switch-like response." So you start very, very slow, and then once you hit an inflection point, suddenly everything shifts. Because it becomes obvious you're on the right direction, everyone is going in that way, and you're getting a lot of benefit from it. And then all your naysayers are stepping back, because now that's obviously where everyone is going, and it's time to get on board.

Rebecca Knight

>> So what does what Russell's describing unlock for the organization? I mean, obviously there's this great spirit of experimentation, and people are able to do things they hadn't been able to do before. But talk a little bit about the opportunity that that provides.

>> I think it's good to maybe think about a concrete example. We can now, as Snowflake, go to a customer who maybe has petabytes of data in a data lake. They have never used Snowflake before, but they can literally attach Snowflake to that data. And we call that "bringing Snowflake to the data," and you can literally show the value of what it means to use the Snowflake engines, and experiences and policies and governance, and all of these different capabilities over that data, literally that day. There is no load. You don't have to suddenly ingest your data. It used to be, like if we were to go back in, say back in time 10 years, the first thing you would do is bring your data, you'd load it into Snowflake, and then you'd be able to begin processing. Well, if you have petabytes of data, the last thing you want to do is move it, copy it, load it, any of those things. All of that has been eradicated by an open table format.

Rebecca Knight

>> I mean, if you think about this, I'm in a customer perspective. Even when I start trying that out, I'm in my mind thinking about, "How am I going to keep these in sync in the future? How am I going to make sure these two different copies are always accurate, that they're up-to-date?" But if there's one copy, that's not even a consideration. You can just start taking advantage of it with no worries. You can stop using Snowflake at any moment, you probably won't, because it's awesome. But you have that option. It's not a lock.

>> Yeah-

Dave Vellante

>> So... Go, please....

>> so we think of that as accessing the data in place. And really what you're seeing is, the central gravity is now like a cloud object storage bucket. And what you're really trying to avoid as an organization is siloing your data, and therefore introducing additional risks in that process, that people have known about but not really kind of internalized. Things like, if I copy the data and I load it into a proprietary system, well then, I have to now manage, say for example, a GDPR concern in two systems, or end systems. How do I know when I've actually solved that problem? I don't. I have to have very, very robust governance controls, that we see customers struggling with all the time and now those things, you have one logical copy of your data. So you are in a much, much stronger position, both for freshness, latency, and security and compliance concerns, with this approach.

Dave Vellante

>> So there's a strong openness theme, obviously, but I want to double click on that because a salient aspect of openness is interoperability. And so, Horizon now implements Polaris APIs. You've got sharing constructs for Iceberg-based data access, that's now inherent. So if we could talk a little bit about the importance of interoperability, some of the specifics of what the community has developed, and how Snowflake supports it.

>> I mean, it's one of our key values in both the Apache Iceberg community, which I'm a PMC member of, the Apache Polaris community that I'm a PMC member of, and we're also working on a new project called Open Semantic Interchange. Which, similarly, is about how do we get these things between people? And the reason that these projects have been successful and they spread so much, is that that group of people who are backing these projects realizes there's only a benefit to this if we remain interoperable. So every discussion is always focused on, "What are we doing? Are we making sure that everyone has consensus?" All the different vendors are looking at this and saying, "Yeah, we can work with that. This is the right next step," so that we never end up with a fork. Because that would be the worst possible scenario for all of the people who work on these projects, like me, very personally biased. Because if there's a fork, what's the point? If one group has their own version, and another group has their own specific version, now they're not interoperable and you might as well just be using a proprietary format. So that drive, that pressure, really makes sure that all of these communities focus completely on making sure that everyone is in agreement before we move forward.

Dave Vellante

>> Yeah. And I would say one of the genius things that the community did, when they moved from just supporting, say for example, Iceberg, the table format, into the API area. Was taking that sort of technology-agnostic approach of using the REST Catalog specification. And the reason for that is, that is agnostic to any implementation vendor choice. And I think, big kudos to the community for making that kind of brave decision, because what that has enabled is all of the kind of vendor implementations of those standards adhere to a technology-neutral based implementation. So I think that was very, very forward leaning from the community, across the board, like across the whole of the Iceberg PMC. So kudos to them, for that.

Dave Vellante

>> And Russell, to your point, monetizing forks, it's an anti-open source pattern, right? You got to find other ways.

Rebecca Knight

>> Yeah. I mean, and I think we're actually at a time now in the industry, where that just doesn't work. People see that you've done it and they're like, "I don't need it to be open source if you have the only supporters of this product. If you're the only ones maintaining and improving it, then why do I care that I can see the code?" That's not the real value. The value for me is that, I have that one copy of my data, everyone can read it. And I don't have to even think about, "Will my next tool, will the next big thing in AI work with my data?" Because if it's in Iceberg right now, it will.

Dave Vellante

>> And I think you're right. I think the industry realized this. When that happens, the committer base just consolidates down to one or two names that are sort of part of the ecosystem, and then it's... People, buyers don't walk, they run from that.

>> Yeah. And similarly, though, other things have been built on top. So the fact that when, with IRC, you also get locally scoped vended credentials, so that the community is not just thinking about the standards, but also the secure interoperability elements, and how that should play out across an ecosystem. Where, I think we fully recognize and are building, in a multi-catalog world. There is no enterprise out there that just says, "Oh, I'm going to have one catalog, or one vendor," or any of these things. And so we're working in that model, of a coexistence model, across these implementations.

Dave Vellante

>> I want to ask you about direction, because you mentioned OSI. I'm trying to connect the dots here. We kind of started out governing the technical metadata, bringing in the operational metadata. Now, with OSI, Cortex Sense, people are talking about context and ontologies, and so forth. Bringing in business process logic, and connecting to other parts. And this is the exciting part of where we're going, the business value on top. Can you give us a little, sort of directional angles, on where you see that going? How organizations are going to be able to bring in some of that tribal knowledge, if you will, and start to automate that?

Dave Vellante

>> I mean, I think that's actually the right perspective. We have this interchange of the data, we have this interchange of access to the data, but the things that we need now in this... I mean, this is hard to say if we'll need it in a year, but right now all of these agents to do really good work. If you want Koka to do the best work it can, if you want Cowork to work as good as it can, it needs the context beyond table names and identifiers. It actually needs to know the concepts that are tied to how your business actually operates. And we've seen, again, this happens all the time, we have a dozen competing variations on how this should actually be stored and exposed. And I mean, I think we're getting ahead of it here, hopefully, by saying, "We're going to work with other groups to say, 'This is how we'll define the exchange. We're going to say these are what the entities look like,'" so that now when your agent goes to a database, it's going to be able to pull in all of that context in a well-defined way, know exactly how to gather it all together. And your agent will be able to go off and run. Because we're kind of getting past that point where, the agent's going to write SQL, but it's going to write it with all of this other information as part of it. So I think that's the first future direction.

Dave Vellante

>> "And I'm talking to my data, too." A text to SQL. Yeah.

Dave Vellante

>> There's the thing, you're taking ownership, we call it this "agency over your data," right?

Dave Vellante

>> Yeah.

>> You're claiming control of the data, and you're using that. But it's really also, I think moving forward, we're seeing things like the importance of data products as the thing that kind of brings, all the things that Russell was talking about together, as a unit of sharing in a producer consumer model. Rather than thinking about data engineering as being like, "I am the creator of pipelines." Pipelines to do what? The future of this space kind of looks more like a data product owner shaping a data product for an organization, making that available to be shared both inter and intra organizationally, and then working back from say a model. Like I was speaking to one financial services company here at Snowflake Summit, and they were saying, "We start from the model and we work back to the data." And they're then using that as their pattern for sharing all the way across their organization. I think that's super exciting and very empowering. And I think for people that are questioning, in this agentic era, what it means for data engineers. I think actually, it's a really exciting time, because it actually does elevate them in the organization. But again, to your point, you have to take those steps.

>> I'd say, the analogy I would bring is, when we see LLMs in coding circumstances, it's pretty obvious. We've gotten to the point where your engineer does not have to write a for loop anymore. That's obviously not a part of a core competency. They're working at this higher level of describing the product, and describing what they want it to do, and maybe some technical aspects of how that works. And the same thing is happening in the data engineering space. We're probably going to be in a year or two away from anyone writing SQL. Instead, you're going to be writing, "These are the relationships between my different types of data, this is how they are structured, this is where you find this information, this is where you find that information." So it'll all just be at a higher level. I think we just keep going up the stack.

Rebecca Knight

>> That higher level is great for people who are seasoned, veteran data engineers, but what would you say to more entry level folks who are seeing the job market change before their eyes? And what kinds of skills, where should they invest their time, I guess I should say.

Dave Vellante

>> I mean, that's a really hard question. And I try to help a lot of folks entering the field right now. I do a lot of mentoring, and I try to talk to them about-

Rebecca Knight

>> Good deed....

>> what to do.

Rebecca Knight

>> Yeah.

>> And really, I try to say, "You need to position yourself really as this oracle in the system. You're no longer the person who's writing code." I mean, again, to give this analogy, there was a time where we didn't have good compilers, and everyone had to be an expert on, "Well, how do I unroll this loop to get the best performance out of this particular micro architecture?"

Dave Vellante

>> Squeeze it.

>> And then 10 years later, that was a nonsense skill. That was something that no one needed to do. But what you still needed to know is, "Well, what was the basic found fundamental thing that was happening in this processor?" So that when I architect at this higher level, I'm aware of the optimizations that are being made. And then, with LLMs, now I think we're just another step above that. "Are these encapsulations of the code correct? Are these patterns that are writing correct?" So I encourage people who are entering the field now to focus on still knowing those basics, but realize that your practice, the thing you should be doing on a day-to-day basis, is evaluating the products that are produced by these models and not by your own hand. So you want to be very familiar, but your job now is a reviewer, and a director. And it's not a typist.

Dave Vellante

>> That's good advice. And I'll follow up with that, because people say, "Oh, learn AI." Well, what does that even mean? What you just described is learn, really, how the fundamentals of the system work and then how to apply AI on top of it. And somebody yesterday, it might have been on theCUBE or in the evening event, said, "And then, figure out what the business needs, and work back from that."

Dave Vellante

>> Exactly.

Dave Vellante

>> "Because that's going to make your AI skills and your coding skills much more powerful."

>> Exactly. You're turning from, say for example, a pipeline which can often be considered a cost, and part of the cogs of running the business, into value creation. And you're trying to elevate people into that business, and thinking about the outcomes, and the value of what it is that they're trying to bring to bear in the organization. So yeah, that fully resonates with what I've seen.

Dave Vellante

>> I mean, the forward deployed engineers that you talk to will tell you that's the most important skill, is really understanding the business and being able to apply the problem.

>> It's always been true, right?

Dave Vellante

>> Yeah, right.

>> We've just taken some of the busy part out, and now that's your whole focus. You are only using your reviewing brain for the mechanical parts, but the business purpose, the design, that's still all mostly on you. Even if you ask agents to do designs for you, if you don't understand what you're actually trying to do, I mean, you're in a sea of hallucinations, right?

Dave Vellante

>> I got to ask you, as we close here, there's still some FUD in the market, about things like Horizon versus Polaris. You guys chose to have really distinct pass, and names, obviously work together. Help us understand the positioning, how you communicate to customers, where I should use each, how they relate to each other.

>> I mean, for me, so I'm the open source guy. I obviously love loading up all my own code, and running my own everything. So there's a bunch of customers who are like me, who are in those ecosystems where they're like, "I don't want to have anything handed over to anybody. So I'm going to run Polaris on my own, going to do my own hosting, I'm going to do this." I mean, maybe they even rent their own servers somewhere, and they're not using cloud service providers. So there is a whole class of folks over there who just want complete control, and they want to be able to dip their toes in Snowflake whenever they want, and those are Polaris users. And then in my mind, there's a shift over. If you're already a Snowflake customer, you already have all your accounts, your ID, everything is set up within the Snowflake ecosystem. We just open the door for all Iceberg usage to you through Horizon. So if you're already comfortable within the Snowflake ecosystem, you don't even have to think about this as a choice. You already have the option right there, and you know that if you ever do want to move, that's also completely available because of how these APIs are structured. I don't know if JRJ has a better answer.

Dave Vellante

>> I would say, the other thing, is that Horizon interoperability is based on Apache Polaris. So it is an open core that is powering your ecosystem. So that also means that it is not like two separate products. It is, you're graduating, as in Russell's example, you're graduating from a self-managed environment to one that is fully managed for you, but on the foundation that is Apache Polaris, so we develop in Polaris and then graduate that into Horizon. And that is a different approach to many of the other vendors that are out there in this space. That you actually have an open core that is present in your managed service. Now of course, on top of that, we then build on all of the other capabilities. Because for a true enterprise kind of metadata catalog, you need to be able to provide all of the security, you need to be able to provide your table maintenance and optimization, and all the governance components, as well as the metastore itself. So I think of it as almost that Polaris is your core, and then you have the value-add that Snowflake is providing on the top, but you have the assurance... Again, not that we ever want this to happen, but if you did decide you wanted to move away from Snowflake-

Dave Vellante

>> You can....

>> you know that you have a foundation that is the same.

>> I mean, I think it shows an amazing amount of trust, that we can open the store. And we know more people are going to come in than are going to go out.

Dave Vellante

>> Because I could choose to, the first group that, I could choose to build that sort of Horizon-like capability on top. If I want to do that. And I would say, by my experience in market analysis that I've done, it's maybe about 15% of the market that will actually choose to do that. They'll maybe regret it, we'll see. But that option is there, and the other point is when you say, "We're not locking you in," you can say, "Here's why." And that has teeth.

Rebecca Knight

>> Yeah. And also, what you're getting to, is also a hub and spoke model.

Dave Vellante

>> Yeah.

>> Because obviously Snowflake is available in all three major public clouds, but there is Iceberg data we know there's petabytes, exabytes of it, that lives in places like on-prem. But with our catalog federation capabilities and everything else, if you are using Polaris to power your, say, on-premises data lake, you can connect that into Snowflake, into the cloud, when you want to be able to share and enable the discovery of that data.

Dave Vellante

>> I'm so glad you brought this up. But go ahead, finish the thought, please.

Rebecca Knight

>> No, no, no. That's fine.

Dave Vellante

>> Because there is a lot of action going on, on-prem. I mean, you could see it in companies like Dell, are obviously resurging, because people don't necessarily want to move all their data into the cloud. They're bringing a lot of the intelligence in. And people say, mistakenly, "Well, Snowflake can't participate in that." But actually, based on what you just described, JRJ, they can-

Rebecca Knight

>> Absolutely....

Dave Vellante

>> actively.

>> Yeah. And we think of it as almost, rather than thinking about the world as, say, data integration. What you're doing is you're connecting to all of your data. And as Benoit was saying in the keynote the other day, the vision of Snowflake was always to be able to work across all data, not just data in Snowflake or tables, it is across all your data. And so as we move forward, it won't just be Iceberg on open kind of table structured data, I'm super excited for all the things the community is cooking up around enabling it for AI use cases and multimodal use cases. So Iceberg is not just for structured data, it is for every type of data.

Dave Vellante

>> And that was always the vision of Iceberg, is bring any compute to any data, and may the best engine win. And I would presume that Snowflake is comfortable competing at that level.

Dave Vellante

>> You're right, we feel very comfortable about that.

Rebecca Knight

>> Yes, indeed. Well, this is a great note to end on. Russell and JRJ, thank you both so much for coming on the show-

Dave Vellante

>> Thanks for having us.

>> Thanks, you guys....

Rebecca Knight

>> and our whole conversation.

>> Thank you.

>> See you next year.

Rebecca Knight

>> Indeed, we will. Rebecca Knight for Dave Vellante. Stay tuned for more of theCUBE's live coverage of the Snowflake Summit, just after a quick break. You're watching theCUBE, the leader in enterprise tech news and analysis.