[Bug]:Unable to find delete file

[LeafAfterglow]

What happened?

Caused by:org.apache.iceberg.exceptions.ValidationException:Missing required files to delete

Affects Versions

0.5.2

What engines are you seeing the problem on?

No response

How to reproduce

No response

Relevant log output

No response

Anything else

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[HuangFru]

Thanks for reporting this bug. Can you describe this problem in detail? Which format? What engines? How to reproduce? Can you show the complete exception stack? And the Amoro community encourages everyone to communicate in English, which will facilitate more people to participate in the discussion.

[LeafAfterglow]

Caused by: org.apache.iceberg.exceptions.CommitFailedException: Base metadata location 'hdfs://jf-iceberg/iceberg/jf_hive_catalog/warehouse/BSS_ODS.db/eda_dw_etl/ord_ord_offer_inst_attr_ext/metadata/54816-959f74aa-f316-4b06-8abc-edd205379302.metadata.json' is not same as the current table metadata location 'hdfs://jf-iceberg/iceberg/jf_hive_catalog/warehouse/BSS_ODS.db/eda_dw_etl/ord_ord_offer_inst_attr_ext/metadata/54818-ef35fe28-72ba-404a-a2f3-c9a0f6eccad0.metadata.json' for bss_ods.ord_ord_offer_inst_attr_ext at org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:219) at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:135) at com.netease.arctic.op.ArcticTableOperations.lambda$commit$0(ArcticTableOperations.java:53) at com.netease.arctic.table.TableMetaStore.call(TableMetaStore.java:220) at com.netease.arctic.table.TableMetaStore.lambda$doAs$0(TableMetaStore.java:200) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at com.netease.arctic.table.TableMetaStore.doAs(TableMetaStore.java:200) at com.netease.arctic.io.ArcticHadoopFileIO.doAs(ArcticHadoopFileIO.java:188) at com.netease.arctic.op.ArcticTableOperations.commit(ArcticTableOperations.java:52) at org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$5(BaseTransaction.java:422) at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413) at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219) at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203) at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196) at org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:418) at org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:302) at com.netease.arctic.op.ArcticTransaction.commitTransaction(ArcticTransaction.java:146) at com.netease.arctic.server.optimizing.UnKeyedTableCommit.replaceFiles(UnKeyedTableCommit.java:234)

[baiyangtx]

Based on the exception information, it appears that during the process of submitting optimization by AMS, there were other submissions that occurred, resulting in a submission conflict. It is likely due to the prolonged submission time. It is possible that there were too many manifest and manifest list files written in a single operation. During the apply snapshot process, the Flink task wrote a new commit.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions[bot]]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'