add AmsAssignService and ZkBucketAssignStore to implement balanced bucket allocation in master-slave mode

[wardlican]

Why are the changes needed?

This implements the logic for dynamically and evenly distributing buckets based on the number of AMS nodes.

Close #3921 .

Brief change log

AmsAssignService Functionality

1. Periodic Allocation: Performs bucket ID allocation every 30 seconds (configurable)
2. Node Change Detection: Detects new and offline nodes
3. Incremental Allocation: Minimizes migration, preserving existing node buckets as much as possible
4. Load Balancing: Distributes buckets evenly across all live nodes
5. Offline Node Handling: Detects offline nodes based on a timeout mechanism and reallocates their buckets
6. Master Node Only: Only the leader node executes the allocation logic.

ZkBucketAssignStore Features

1. Storage Implementation: Stores bucket assignment information based on ZooKeeper.
2. Data Storage:
   • List of bucket IDs for each node (JSON format)
   • Last update timestamp for each node
3. CRUD Operations:
   • Save/Update Node Assignments
   • Query Node Assignments
   • Delete Node Assignments
   • Query All Node Assignments
4. Access Control: Only the leader node can save/delete assignment information.
5. Path Structure: /{cluster}/amoro/ams/bucket-assignments/{nodeKey}/assignments and /{cluster}/amoro/ams/bucket-assignments/{nodeKey}/last-update-time

two classes that work together to implement dynamic bucket ID allocation and storage in master-slave mode.

How was this patch tested?

• [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

• [ ] Add screenshots for manual tests if appropriate

• [ ] Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[codecov-commenter]

Codecov Report

:x: Patch coverage is 71.01770% with 131 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 29.22%. Comparing base (4382d55) to head (f2ecc06). :warning: Report is 31 commits behind head on master.

Files with missing lines	Patch %	Lines
...java/org/apache/amoro/server/AmsAssignService.java	78.16%	35 Missing and 15 partials :warning:
...a/org/apache/amoro/server/ZkBucketAssignStore.java	66.66%	34 Missing and 5 partials :warning:
...org/apache/amoro/server/AmoroServiceContainer.java	11.53%	22 Missing and 1 partial :warning:
...apache/amoro/server/HighAvailabilityContainer.java	70.68%	13 Missing and 4 partials :warning:
...a/org/apache/amoro/properties/AmsHAProperties.java	0.00%	2 Missing :warning:

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3922      +/-   ##
============================================
+ Coverage     29.00%   29.22%   +0.22%     
- Complexity     3850     3915      +65     
============================================
  Files           631      630       -1     
  Lines         50770    50997     +227     
  Branches       6543     6518      -25     
============================================
+ Hits          14724    14906     +182     
- Misses        34989    35039      +50     
+ Partials       1057     1052       -5     

Flag	Coverage Δ
core	29.22% <71.01%> (+0.22%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions[bot]]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[wardlican]

Please conduct a code review of the implementation here.