amoro icon indicating copy to clipboard operation
amoro copied to clipboard

[Bug]: create an unused .keep file in mixed_hive format table

Open nicochen opened this issue 7 months ago • 4 comments

What happened?

It creates millions of '.keep' small files in my case and there is no mechanism to clean it up. It seems like '.keep' file is created and never been used. Perhaps it should not been created or implement an orphan clean logic to clean it up. Image

Affects Versions

0.7

What table formats are you seeing the problem on?

Mixed-Hive

What engines are you seeing the problem on?

Optimizer

How to reproduce

No response

Relevant log output


Anything else

No response

Are you willing to submit a PR?

  • [x] Yes I am willing to submit a PR!

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

nicochen avatar May 06 '25 07:05 nicochen

@Aireed @baiyangtx PTAL

nicochen avatar May 06 '25 07:05 nicochen

@Aireed @baiyangtx PTAL

yes, as discussed offline, we can move the condition of whether the hive data changes forward and apply it to whether we need to create a catalogue or not

Aireed avatar May 09 '25 03:05 Aireed

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

github-actions[bot] avatar Nov 06 '25 00:11 github-actions[bot]

could you take a look? @zhoujinsong @xxubai

turboFei avatar Nov 06 '25 01:11 turboFei