Support validate and related logic in Snapshot
Is your feature request related to a problem or challenge?
To detect conflicts, we need to have validate and related logic to scan manifests and validate against history.
References:
- java: https://github.com/apache/iceberg/blob/c2478968e65368c61799d8ca4b89506a61ca3e7c/core/src/main/java/org/apache/iceberg/BaseRowDelta.java#L128
- python: https://github.com/apache/iceberg-python/pull/1935
Describe the solution you'd like
Add related logic to SnapshotProduceOperation so different actions like RowDelta, OverwriteFiles, and RewriteFiles can use and override (if needed)
Willingness to contribute
I would be willing to contribute to this feature with guidance from the Iceberg Rust community
predicate/residual evaluator and manifest filter are also important (to reduce conflict rate) but they could be completed separately, wdyt? @Xuanwo @liurenjie1024 @jonathanc-n
Yes those can be completed separately, I will try to get residual evaluator done (a bit difficult since the schema + filters in rust are a bit different)
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.