helix icon indicating copy to clipboard operation
helix copied to clipboard

Creating emergency rebalance pipeline

Open NealSun96 opened this issue 2 years ago • 0 comments

This is a WIP PR for creating the emergency rebalance pipeline. NOTE: this is not intended to be merged, but will be used for benchmarking performance.

Logic flow: Case 1: when a rebalance is triggered, and there's no illegal placement (no new resources, no node down):

  1. Emergency rebalance pipeline is triggered.
  2. Emergency rebalance triggers partial rebalance if no partial rebalance exists; else it does nothing.
  3. Partial rebalance triggers asynchronous calculation.
  4. Emergency rebalance returns the existing Best Possible assignment.
  5. If partial rebalance calculates a best possible assignment that's different from the existing one, it persists the result into ZooKeeper, and triggers the pipeline again; else it does nothing.

Case 2: when a rebalance is triggered, and there are illegal placements:

  1. Emergency rebalance pipeline is triggered.
  2. Emergency rebalance assigns all illegal placements correctly, cancels existing partial rebalance thread, and persists the result to ZooKeeper.
  3. Emergency rebalance triggers partial rebalance.
  4. If partial rebalance calculates a best possible assignment that's different from the existing one, it persists the result into ZooKeeper, and triggers the pipeline again; else it does nothing.

NealSun96 avatar Aug 09 '22 00:08 NealSun96