iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Support bulk remove orphan files

Open jackye1995 opened this issue 1 year ago • 3 comments

Feature Request / Improvement

Under the case where multiple tables share the same root location (e.g. same bucket with object storage mode enabled), we need a bulk remove orphan files procedure to take into consideration of files in multiple tables instead of just one table.

The assumption is that all tables in this bulk operation share the same prefixes, and no other table data are within the specific prefixes.

The list of table can be manually listed, or from a namespace, or from all namespaces. Those can be separated add-on features.

Query engine

None

jackye1995 avatar Mar 15 '23 04:03 jackye1995