amoro icon indicating copy to clipboard operation
amoro copied to clipboard

[Improvement]: For Mixed-Hive tables, default refresh time for Hive partitions should be configured

Open shendanfengg opened this issue 2 years ago • 1 comments

Search before asking

  • [X] I have searched in the issues and found no similar issues.

What would you like to be improved?

image As shown in the figure, our default configuration is -1, which may result in Full Self-Optimizing not being executed for some Mixed Hive table history partitions.

How should we improve?

Set a default value for this configuration

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

shendanfengg avatar Nov 30 '23 09:11 shendanfengg

@shendanfengg I saw you trying to change the default value of base.hive.refresh-interval in order to refresh the data in Hive for all Mixed-Hive format tables every half an hour by default.

This configuration was initially disabled because we couldn't ensure that all users of Mixed-Hive format tables have this requirement. Can you adjust the default configuration under Catalog to enable this feature for all your Mixed-Hive tables instead of directly changing the default value of this configuration? I still can't confirm that all users of Mixed-Hive format tables have this requirement.

Hi, danfeng, please check the comment in releated PR

majin1102 avatar Mar 14 '24 01:03 majin1102

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

github-actions[bot] avatar Sep 11 '24 00:09 github-actions[bot]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'

github-actions[bot] avatar Sep 25 '24 00:09 github-actions[bot]