ZENOTME

Implement the equality delete writer

5

After we finish https://github.com/apache/iceberg-rust/pull/275, we can implement the equality delete writer based on this framework. There is a [rust implementation](https://github.com/icelake-io/icelake/blob/main/icelake/src/io_v2/base_writer/equality_delete_writer.rs) that can be referred to in icelake. But better design...

field-id in avro schema is missing

7

According https://github.com/risingwavelabs/risingwave/issues/14106, avro-rs don't serialize the field-id. We will close this issue after the fix be accept by upstream and release it.

Implement the equality delta writer

5

After #340 #341, we can implement the [equality delta writer](https://github.com/apache/iceberg/blob/e3b78be9ac32533500136e1a32d7f02efc03bef8/core/src/main/java/org/apache/iceberg/io/BaseTaskWriter.java#L108). There is [a rust implementation](https://github.com/icelake-io/icelake/blob/main/icelake/src/io_v2/functional_writer/equality_delta_writer.rs) in icelake can be refer

Inspired by https://github.com/apache/iceberg-rust/pull/275#issuecomment-2041304545, I created this issue to track all our write task. It's based on [doc](https://docs.google.com/document/d/1YncDX-qQ1T9jBGQmJNtRcPU1trRi00cB8eykv5diKw4/edit?usp=sharing). - [ ] https://github.com/apache/iceberg-rust/issues/345 - [ ] https://github.com/apache/iceberg-rust/issues/329 - [ ] Overwrite table...

Implement partition writer

8

Implement two partition writers: - [ ] fanout partition writer: partition in writer - [ ] partition writer: write the data already partitioned

Implement the position delete writer

5

After we finish #275, we can implement the position delete writer based on this framework. There is a [rust implementation](https://github.com/icelake-io/icelake/blob/main/icelake/src/io_v2/base_writer/position_delete_writer.rs) that can be referred to in icelake. But better design...

Support to append file on table

1

After we finish https://github.com/apache/iceberg-rust/pull/275, we can support to write data and generate the`DataFile`. To complete #329, the next job is to support append data file and commit on transaction. We...

Inheriting the sequence number from manifest list when load manifest

3

ref: https://iceberg.apache.org/spec/#sequence-numbers

Integrate with datafusion

4

After support basic scan and catalog, we can consider to integrate with datafusion to speed up data driven tests.

For now, we can't suppor ser/de bytes-like type well because need some fix/feature provided by avro-rust: uuid, bytes, fixed, decimal - [x] support bytes, fixed, decimal(https://github.com/apache/avro/pull/2567) - [] support custom...