delta
delta copied to clipboard
[WIP][Kernel] Support enabling RowTracking
Which Delta project/connector is this regarding?
- [ ] Spark
- [ ] Standalone
- [ ] Flink
- [x] Kernel
- [ ] Other (fill in here)
Description
This PR is trying to unblock RowTracking feature in Kernel, which includes
- Unblock the row tracking feature enablement in the
TableFeatures. - Support client passing the rowIdHighWatermark when creating the transaction. We would do enough check to make sure that all rows would not exceed this provided high watermark, then we would set this row id high watermark in this txn.
- Allow setting the these table properties
delta.rowTracking.materializedRowIdColumnNameanddelta.rowTracking.materializedRowCommitVersionColumnName. - Allow passing baseRowId and defaultRowCommitVersion when generate
IcebergCompatWriterV1AddActionandIcebergCompatWriterV1RemoveAction
How was this patch tested?
Existing and newly added unit tests.