delta icon indicating copy to clipboard operation
delta copied to clipboard

[WIP][Kernel] Support enabling RowTracking

Open KaiqiJinWow opened this issue 5 months ago • 0 comments

Which Delta project/connector is this regarding?

  • [ ] Spark
  • [ ] Standalone
  • [ ] Flink
  • [x] Kernel
  • [ ] Other (fill in here)

Description

This PR is trying to unblock RowTracking feature in Kernel, which includes

  1. Unblock the row tracking feature enablement in the TableFeatures.
  2. Support client passing the rowIdHighWatermark when creating the transaction. We would do enough check to make sure that all rows would not exceed this provided high watermark, then we would set this row id high watermark in this txn.
  3. Allow setting the these table properties delta.rowTracking.materializedRowIdColumnName and delta.rowTracking.materializedRowCommitVersionColumnName.
  4. Allow passing baseRowId and defaultRowCommitVersion when generate IcebergCompatWriterV1AddAction and IcebergCompatWriterV1RemoveAction

How was this patch tested?

Existing and newly added unit tests.

Does this PR introduce any user-facing changes?

KaiqiJinWow avatar May 20 '25 00:05 KaiqiJinWow