Ryan Blue

Merged! Thanks @cashmand for getting this in. Nice work!

@amogh-jahagirdar, @hantangwangd, I'm not sure that incremental cleanup is doing anything wrong here. Incremental cleanup deletes data files when the snapshot that removed them from the table is expired. If...

I thought this was already supported, but I don't see it. The way we did this at Netflix was to add a table-specific property to SQL, like `spark.sql.iceberg.db.table.split-size=...`. This is...

I think this is a good idea. I'll put it in my queue to review.

@nastra, where are we with the fix for serialization here?

Oops, I didn't mean to close this! I want to work on getting it in next

@zhongyujiang can you please add more to the description about what is included here and how you solved the problems with record materialization?

@itaise, it looks like you're saving the table as Parquet, not Iceberg. That could be a problem.

I think this is fine. @danielcweeks is the organization what you want?

While I think the goal here is a good one, the implementation looks too complex to be workable in its current form. The primary issue that we currently have is...