Discussion: S3 soft deletion attributes

[c-thiel]

Currently our S3 FileIO does not support s3.delete.* attributes that are used to implement soft-deletion with a custom lifecycle policy.

Follow-up of https://github.com/apache/iceberg-rust/pull/505

[Xuanwo]

Copied my comment below:

The Iceberg Java FileIO implements soft deletes using PutObjectTags and a user-configured lifecycle to support it. Deleted objects can still be scanned or listed because S3 does not recognize the concept of soft deletes. While it's possible for OpenDAL to support this, I doubt its usefulness.

I personally think adopting the object version to implement soft-delete is a better idea for that.

[liurenjie1024]

I'm not an s3 expert, but s3 docs shows that both custom tags and versioned object are two ways to allow custom lifecycle management, right? I'm not sure about the motivation behind the choice of using tags rather than versions. Maybe we could ask for help in dev list for the importance of supporting this feature?

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions[bot]]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'