Szehon Ho

@RussellSpitzer @aokolnychyi Rebased the pr , it's still using the manual way to filter out snapshots from all_manifest table, if you guys have time to take a look. The idea...

@rdblue yea thanks, I realized it after asking.

Ref : discussion on https://github.com/apache/iceberg/pull/4736. This implements the original idea to reduce the 'deleteCandidate' scan to just the ones from deleted snapshots. There is another idea to remove current manifests...

Update: I implemented a draft for removing current manifests from delete candidate calcuation, but found it doesn't help much and actually makes the performance worse: see https://github.com/apache/iceberg/pull/4736#issuecomment-1176854697. , so I'll...

Test got cancelled somehow, retriggering

@zhaomin1423 actually do you mind doing the same change for other spark versions? I feel normally we do feature for specific spark version(s), but for something like this that's purely...

@zhaomin1423 yea its a good question, it seems it was done this way in the initial split of the Spark modules to different versions. I think most of the spark...

I think that's a great idea, to have the Catalog's regiserTable just first call dropTable (purge=false) , instead of throw an error, if table exists, it could be a very...

So still to me it makess sense to add a force option to registerTable. Initially we were debating between adding a new Catalog API , or re-using registerTable with force...

> I prefer this approach (I think I said so before :) ) Yes fine you win :). Yea the only kind of downside is if we error re-creating the...