starrocks
starrocks copied to clipboard
Support iceberg v2 table
Enhancement
v2 format can support update for iceberg table related with #1030
We will use c++ to implement the merge on read logic
Phase 1: Key points
- Only support Parquet and Orc
- Schedule data file scan range together with related delete files, process them together on BE
- Support position pushdown separately for Parquet and Orc
- Only support position delete file
- [ ] Support get delete file information in Planner and send to BE
- [ ] Parquet: Support virtual column _POS for position delete file
- [ ] Parquet: Support position pushdown
- [ ] Orc: Support virtual column _POS for position delete file
- [ ] Orc: Support position pushdown
- [ ] Implement merge on read logic
Phase 2x:
- [ ] Support equality delete file
- [ ] Add Avro reader
cc @choury @DorianZheng @mxdzs0612 @stephen-shelby
We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!