Snowflake Summit 2025 | Russell Spitzer, Snowflake & Ankit Singh, Affirm

Clips
More from Snowflake Summit 2025

Russell Spitzer

Principal Software Engineer

Snowflake & Apache Polaris PPMC

Ankit Singh

Senior Staff Software Engineer

Affirm

play_circle_outline Iceberg: The Foundation of Affirm's Financial and Data Auditing with Time Travel Capabilities

play_circle_outline Snowflake's Open Catalog and externally managed Iceberg tables for data storage

play_circle_outline Polaris extending governance and security to data managed in open formats

play_circle_outline Focus on giving users choice of compute engine based on skill sets

Info
Transcript

Russell Spitzer, Snowflake & Ankit Singh, Affirm

Russell Spitzer

Principal Software Engineer Snowflake & Apache Polaris PPMC

Ankit Singh

Senior Staff Software Engineer Affirm

In this conversation at Snowflake Summit 2025, Russell Spitzer, principal software engineer at Snowflake, and Ankit Singh, technical leader for data storage & services at Affirm, join theCUBE’s Dave Vellante and John Furrier to explore how open data strategies are reshaping financial workloads in the AI Data Cloud. Broadcasting live from the show floor, the interview spotlights Snowflake’s expanding role as the connective tissue for AI-driven innovation, data collaboration and operational efficiency.

Singh explains how Affirm uses Apache Iceberg and t... Read more

explore Keep Exploring

What role does Iceberg play in the financial pipelines and auditing process of the company? add

What workloads go where within the context of Iceberg and Snowflake for an organization using Spark as their primary choice for compute? add

What is the reason behind the creation of Apache Polaris? add

What is a key aspect of Iceberg according to the text provided? add

bolt Powered by CUBE AI

Russell Spitzer, Snowflake & Ankit Singh, Affirm

search

Rebecca Knight

>> Good afternoon everyone, and welcome back to theCUBE's live coverage of the Snowflake Summit 2025 here at the Moscone Center. I'm your host, Rebecca Knight, alongside my co-host and analyst George Gilbert. And I would like to welcome two new guests to the show. We have Russell Spitzer, he is Principal Software Engineer, Snowflake and Apache Polaris at Snowflake. Thank you so much for coming on Russell. And Ankit Singh, Senior Staff Software Engineer at Affirm. Thank you so much and we're delighted to have you on the show. Ankit, I'm going to start with you and have you just describe to our viewers a little bit about Affirm. I mean, I think a lot of people know, founded by PayPal founders. So tell us and our viewers about what you

Ankit Singh

>> Affirm started with a very simple mission, build honest financial products. Our goal was build a product that allows people to exactly know what they're paying for and how much it's going to cost over time. Honest financial product basically means for us, we never gain from our users' misfortune. So there's no late fees, there's no hidden charges, and we've stuck to that and that isn't changing. And our founder, CEO Max, and he's still involved day to day with everything, so we still continue to grow and expand, yeah.

Rebecca Knight

>> And how is Affirm using Apache Iceberg and Snowflake's open catalog? What does it look like in practice?

Ankit Singh

>> Iceberg is our backbone for all financial pipelines. As a financial company, we have strict obligations not just to our users, but also our banking partners, our lenders, our loan sales buyers. Iceberg provides us this unique data lake opportunity where we can aggregate the view of all the transactions we do with our users, all the interactions our users have with our financial systems, and essentially time travel to a point where we can rebuild our understanding of how money movement happens, how we do capital funding, how our pipelines are performing over time. And being in the FinTech industry, we are obligated to provide quick answers to auditors. With Iceberg, we are able to time travel back pretty quickly and we've managed to change our audit windows from months and weeks to minutes now. So Iceberg basically plays the primary storage for our financial platform.

George Gilbert

>> So within the context of Iceberg, there's the externally managed Iceberg managed by the Polaris Open Source catalog, and then there's managed Iceberg, managed Iceberg within the Snowflake Horizon catalog and the DBMS and native tables. So what workloads go where?

Ankit Singh

>> Our primary choice for compute is Spark. So we publish all our data sets into our own storage layer S3 bucket, and we use Snowflake's Open Catalog. So all of our initial storage goes into initial data sets, go into externally managed Apache Iceberg tables. Transform data sets for internal analytics experimentation, we use Snowflake computer engine to publish into Snowflake managed tables. And the federation between Snowflake catalog and Open catalog allows us to write once and read everywhere.

Rebecca Knight

>> Russell, I want to bring you in here to the conversation-

Russell Spitzer

>> Of course.

Rebecca Knight

>> Because we know that governance and security are top in mind for financial institutions. It's a highly regulated industry, we know that. So how is Snowflake extending those protections to data managed in open formats like in Iceberg?

Russell Spitzer

>> Yeah, I think that's one of the reasons that Apache Polaris exists, is that we're looking for a place where we can actually put those decisions into an open platform where everyone can make decisions and everyone can use them inter operably. We have a lot of users whose Snowflake is not their only engine, and we want to make sure that they get the same experience regardless of where they're coming from. So when I think about Polaris, it's about how do we take all those great governance features that only exist in Snowflake and make them available to everyone? We want to make sure that everyone who's using their iceberg tables, regardless of what engine you're choosing, gets the same experience.

George Gilbert

>> So let's talk about that because that's a moving target from what we understand.

Russell Spitzer

>> Oh, definitely.

George Gilbert

>> But moving target in a good sense, in that it started out when it was announced last year, just technical metadata and I think permissions, but that we're told all the capabilities, as you alluded, in Horizon are migrating over time. Horizon will always be the richer governance solution and the user experience. But tell us about what level of functionality has migrated to Polaris and how it's synchronized with Horizon so that you're governing your entire Snowflake and Iceberg data estate in a common way. But what level of functionality moves? Where are we now and where will we be in 12 months?

Russell Spitzer

>> I mean, I say we're right at the beginning right now. I mean, the kinds of things that we can put inside of Polaris right now are pretty broad-based access control policies. Someone can read, someone can write, what principles and roles do they belong to and all that sort of great stuff. But where we're looking in the future is we really want to be as fully featured as we can be. And obviously this is a process and we'll have to decide on a case by case basis, what kind of things make sense for everyone to have in a portable way. But one of the things we're working on right now is fine-grain access control. So that's basically getting from beyond the fact that I as a have access to a table and instead saying, I as a user only have access to specific rows within that table. And making that into a portable standard is extremely tricky. But luckily we're able to collaborate because it's an open source project with folks from a variety of different companies representing a raw variety of different engines, to make sure we make something that everyone can kind of agree on and give users the common experience regardless of where they're coming from.

Rebecca Knight

>> And so Ankit, describe how that works within a firm. I mean, I'd love to hear about some of the data or infrastructure challenges that you're having, some of the pain points, some of the things that keep you up at night and how Iceberg has helped you solve them.

Ankit Singh

>> I would want to agree with Russell and fine-grained access control, that's something that always keeps us up at night. Because one thing we'd love to see is a unified mechanism to define a policy and a policy engine that allows us to translate how we manage our cloud infrastructure with data infrastructure, with open catalog infrastructure, specifically to Iceberg. The biggest advantages that we've seen in our operational and analytical workloads are our ability to time travel and the right audit publish pattern that Iceberg does so well compared to other open formats. Our ability to audit data sets before publishing it to our stakeholders. Because when it comes to money movement, you cannot make mistakes. You can be slow but never wrong. That has essentially allowed me to sleep without waking up at night. Trying to fix quality issues, it auto corrects itself and if there is an issue we can remediate and we can be slow. We're never wrong in our money movement pipeline. So that's been the biggest a-ha moment I would say with Iceberg is our ability to branch and audit.

Rebecca Knight

>> getting better sleep, so-

Russell Spitzer

>> I mean, that's my real goal. They say that the key virtue of being an engineer is to be lazy. You want to make sure that anyone working with your software doesn't have to do a lot of work, so...

Rebecca Knight

>> Good point.

Russell Spitzer

>> But how do you rethink about how you allocate workloads between compute engines, where Snowflake is, that has always had that integrated simplicity, but now you also have the option essentially to own your data, where Snowflake doesn't own it. So how does that change your trade-off between all the workloads you might do within Snowflake for that simplicity, and then when might you bring in a specialized engine that just can do something Snowflake can't?

Ankit Singh

>> It depends on our users, I would say. We have a very good amount, a large chunk of our users who are very comfortable with SQL and they do prefer Snowflake. They prefer Snowflake's platform to do analysis, run Cortex AI workloads. And then we have our machine learning engineers, our data engineers, who prefer writing in Spark. So we, like as Russell would say, we are hearing our customers who are engineers within our firm building, for our customers. We are able to now give them the choice of compute engine that works with their skill sets without trying to dictate fine grain, use Spark over Flink, Flink over Trino or anything like that. This has changed our approach in terms of focusing on governance data quality, versus trying to figure out which compute engine would work the best.

Russell Spitzer

>> Yeah, I think this is a key aspect of Iceberg, is you basically just open everything up and users can decide what works best for them. And I mean, I hope that we can win over all workloads eventually, but if you have the option to choose anything, you can get that foot in the door and actually try something new.

Rebecca Knight

>> And allow you all to focus on, as you said, the real value add in terms of security and governance. So Russell, are there any use cases or workloads that maybe aren't getting a lot of attention but that you think really could be game changers, especially as Iceberg and other open technologies really evolve?

Russell Spitzer

>> Oh, that's a great question. I think, well, we're at a big summit talking about AI all the time, and obviously that's one of the spots where I think a lot of folks are paying attention right now, especially within Iceberg. How do we make it work better for AI formats? And the data for AI looks a little bit different than the stuff that we've usually been working with because we deal with these extremely wide tables that represent feature sets, and all of this internal engineering. And the way that we always thought about tables before didn't really consider that we would have these super wide tables that you occasionally add a whole column to or you remove a column from. So there's a lot of work happening right now to make those workloads work better in these table formats. So that's something I'm focused on a lot.

George Gilbert

>> Okay. So what net new data would you apply Snowflake workloads to that are in Iceberg, that you wouldn't have brought into the Snowflake platform before? In other words, now that you have more of your data estate accessible to Snowflake, because you don't have to ingest it into the platform and transform it, what new things might you apply?

Ankit Singh

>> One example I can think of is logging injection. We were very careful about logging injection, given the data security aspect of what raw logs can continue and we would do on read parsing using Snowflake's variant support. Now with Iceberg, we now have long-term storage parsed structured log, data which we on demand mirror our catalog sync with Snowflake. This has changed how engineers debug incidents, how they look at historical data. We always have this choice between write a promQL query to see what's happening today or write something SQL to see what happened over time, and actually zone into things that you wouldn't be able to debug. So logging story, long-term log storage with Iceberg and Snowflake has been a big win for us in terms of how we are able to drive value on non-analytical data sets, but more operational data sets.

George Gilbert

>> Okay, so this is something that might've been a Spark-based workload or an Elastic maybe. But now because you have the integrated simplicity and you don't have to ETL the data into a new platform, so this is a net expansion.

Ankit Singh

>> Yes. One other big use case for us is, we work in the financial industry and most of our processing is a daily basis how banks work here. But we don't have to follow that when we process data. With Iceberg time travel and tags, we can incrementally process information, publish data sets, and then when our partners need the information, we can tag and publish them. So the ability to use branching and tagging with the rollback support on a table level has just allowed us to explore data engineering patterns. That was not possible in the OLAP world before, at least based on what I was doing. But now you can basically mimic a two-phase commit that were native to distributed systems without thinking about it, enabling data engineers to do complex transactional operations that Iceberg just makes it simple for you.

George Gilbert

>> Okay.

Rebecca Knight

>> Excellent. So Russell, you've talked a little bit about some of the new use cases and workloads that you're excited about, and I'm curious about other areas of innovation that we're going to be seeing in the coming months or years that you are looking forward to.

Russell Spitzer

>> Yeah. Well, we just finished Iceberg table spec version three, which has been a huge accomplishment for the community. And of course just a few minutes after we finished that, people started out with their V4 suggestions and proposals, because we have a lot of work to do in the format. Like I said, we want to be better equipped to handle AI and streaming date use cases. We really want Iceberg to end up being the only format you need and have everyone be able to work with it without any kind of ETL. So in general, we're looking towards that future to handle streaming use cases and AI use cases a lot better. I imagine that we'll see a lot of things in the near future probably around changing our metadata layer to make it faster to plan new queries and handle those kind of quick ingest use cases. But we'll see, it's going to be a busy year.

Rebecca Knight

>> And Ankit, anything on your wish list that you're hoping for in V4?

Ankit Singh

>> Cross-table transactions?

Russell Spitzer

>> Yes, we-

Rebecca Knight

>> From your mouth to...

Russell Spitzer

>> Do you have the proposals right now? Engineers at Snowflake are actually working on that at this moment. We are talking with the community and hopefully we'll have something done in the next few weeks.

Rebecca Knight

>> Excellent, wow. Well both, thank you both so much for coming on the show, Russell and Ankit.

Ankit Singh

>> Thank you.

Rebecca Knight

>> A pleasure having you on. I'm Rebecca Knight for George Gilbert. Stay tuned for more of theCUBE's live coverage of Snowflake Summit 2025. You're watching theCUBE, the leader in enterprise tech news and analysis.