amoro icon indicating copy to clipboard operation
amoro copied to clipboard

[Bug]: Cyclic rewrite of a single pos-delete file

Open xxubai opened this issue 1 year ago • 0 comments

What happened?

Optimizer in only one Datafile and pos-delete file need to be rewritten, it may happen that the pos-delete is deleted after being rewritten over and over again: the specific RewriteInput case is as follows: image The file: s3://wap-udp-calling-prod-useast1/wap_ci_usa_users_history/data/orgid_bucket=5/00036-657-872c6863-916b-46c5-911d-ad418d93570c-00001-deletes.parquet will be rewrite a new pos-delete file and then remove in next snapshot.

image image

Affects Versions

master/0.7.x

What table formats are you seeing the problem on?

Iceberg

What engines are you seeing the problem on?

Optimizer

How to reproduce

No response

Relevant log output

No response

Anything else

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

xxubai avatar Oct 23 '24 05:10 xxubai