[HUDI-4582] Support batch synchronization of partition to hive metastore to avoid timeout with --sync-mode="hms" and use-jdbc=false

[honeyaya]

Change Logs

Describe context and summary for this change. Highlight if any code was copied.

What is the purpose of the pull request

when we try to sync 11w partitions to hive by using HiveSyncTool(--sync-mode="hms" and use-jdbc=false) with timeout error.

With https://issues.apache.org/jira/browse/HUDI-2116, this only solved --sync-mode = jdbc with the parameter: HIVE_BATCH_SYNC_PARTITION_NUM, and I want to extend this to hms mode.

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[honeyaya]

@hudi-bot run azure

@hudi-bot run azure

run azure

[honeyaya]

@hudi-bot run azure

@hudi-bot run azure

run azure

[honeyaya]

@jsbali @yihua could you help review this pr, thanks

[honeyaya]

@hudi-bot run azure

@hudi-bot run azure

run azure

[xushiyan]

@honeyaya please also simplify the pr title and add details in change logs section.

[hudi-bot]

CI report:

• 96040212b1e5607f6822e217e81fbcee7cf79af7 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build