amoro icon indicating copy to clipboard operation
amoro copied to clipboard

[Bug]: keyed mixed iceberg format table always failed to expire snapshots of it's basetable

Open liaoyt opened this issue 3 weeks ago • 0 comments

What happened?

In a MixedTable (composed of BaseTable and ChangeTable), both tables share the same table metadata. When performing snapshot expiration:

  1. The change table's snapshot cleanup completes and commits, which updates the shared table metadata (incrementing the meta_version).
  2. The base table's snapshot cleanup then attempts to commit, but it still holds a stale reference to the table metadata with an outdated meta_version.
  3. This causes the base table's commit to fail with a meta_version conflict error, since the metadata version has already been incremented by the change table's commit.

Root Cause: The BaseTableMaintainer and ChangeTableMaintainer both operate on the shared table metadata, but when changeMaintainer.expireSnapshots() commits, it updates the metadata version. The baseMaintainer instance still holds the old metadata version, leading to a conflict when it tries to commit its snapshot expiration.

Expected Behavior: After the change table's snapshot cleanup commits, the base table should refresh its metadata reference to get the latest version before attempting its own snapshot cleanup commit.

Affects Versions

master

What table formats are you seeing the problem on?

Mixed-Iceberg

What engines are you seeing the problem on?

AMS

How to reproduce

  1. create mixed iceberg table and writon with flink cdc
  2. enable ams auto expire snapshots with default setting (ttl can be shorter for reproduct)
  3. after expiring process finished, change table success remove snapshots and base table keeps all the snapshots

Relevant log output


Anything else

No response

Are you willing to submit a PR?

  • [x] Yes I am willing to submit a PR!

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

liaoyt avatar Dec 10 '25 02:12 liaoyt