cruise-control icon indicating copy to clipboard operation
cruise-control copied to clipboard

Add endpoint to move all replicas from a disk to other disks of the same broker

Open ilievladiulian opened this issue 3 years ago • 0 comments

This PR resolves #1907 .

This PR introduces a new endpoint that moves replicas from a specified disk to other disks of the same broker. This will allow the removal of the disk without the loss of data.

The replicas are moved in a round-robin manner to the remaining disks, from the largest to the smallest, while checking the following constraint:

1 - (remainingUsageAfterRemoval / remainingCapacity) > errorMargin

where

remainingUsageAfterRemoval = current usage for remaining disks + additional usage from removed disks
remainingCapacity = sum of capacities of the remaining disks
errorMargin = configurable property (default 0.1); it makes sure that a disk percentage is always free when moving replicas

What is new

The new endpoint is available as follows:

POST /remove_disks?brokerid_and_logdirs=[brokerid1-logdir1,brokerid2-logdir2...]

What is missing

After all replicas are moved from the specified disk, the disk may still be used by CC during rebalances and Kafka can still use it when creating topics.

ilievladiulian avatar Sep 19 '22 15:09 ilievladiulian