apache
feat: Add deletion vector related fields in spec types
Which issue does this PR close?
This PR sync up with upstream iceberg spec on deletion vector related fields. Reference PR: https://github.com/apache/iceberg/pull/11240 Reference file: https://github.com/apache/iceberg/blob/main/format/spec.md#manifests
Context: I'm working on iceberg related projects in my company, one of the features I'm working on is deletion vector. Because there's no existing rust support for it, I need to do some of the work my own (i.e. deletion vector read and write).
My philosophy is to reduce discrepancy with upstream is:
- Avoid touching existing code unless unavoidable, and collaborate with open community if possible;
- For deletion vector issue, I could make no other changes and keep puffin write/read completely change on my end;
- The schema change in this PR is something hard to work-around. :(
I personally think the schema change alone (without actual deletion vector feature implementation) make sense, because it's a sync-up from iceberg spec sync (which has already been acknowledged by the community), as the PR.
I'm happy to discuss more on this PR, I'm willing to provide more on the motivation, my work-around and followup; I'm also willing to contribute if the community requires more hands on it.
Are these changes tested?
This PR is a no-op change, I confirm I could build + link with no problem.
Hi @dentiny, I have updated the PR title to better reflect the changes. I have to say that I completely agree with your "upstream first" philosophy, and I appreciate any kind of help. Welcome to join the community!
