Alexey Zatelepin

Flex partition assignment stage 1

3

Introduce `shard_balancer` and `shard_placement_table` and use them to drive reconciliation of partition placement across shards. Cross-shard movements are executed with the new “push” protocol. The assignments themselves still come from...

[WIP] Flexible partition core assignment dev

13

## Backports Required - [ ] none - not a bug fix - [ ] none - this is a backport - [ ] none - issue does not exist...

c/controller_backend: bootstrap from shard_placement_table

3

When bootstrapping controller_backend, we don't really need to try and reconcile all ntps in the cluster, we can start with the set that is in shard_placement_table (i.e. all ntps with...

c/shard_balancer: assign shards concurrently

2

For now, when assigning shards only involves modifying in-memory state, we can get away with assigning each ntp sequentially. But in the future it will also involve kvstore writes (with...

Backport of PR https://github.com/redpanda-data/redpanda/pull/18021 Fixes https://github.com/redpanda-data/redpanda/issues/18040

CI Failure (failed to download redpanda release) in `UpgradeToLicenseChecks.test_basic_upgrade`

1

https://buildkite.com/redpanda/redpanda/builds/39601 ``` Module: rptest.tests.license_upgrade_test Class: UpgradeToLicenseChecks Method: test_basic_upgrade Arguments: { "cloud_storage_type": 1 } ``` ``` test_id: UpgradeToLicenseChecks.test_basic_upgrade status: FAIL run time: 91.594 seconds RemoteCommandError({'ssh_config': {'host': 'docker-rp-1', 'hostname': 'docker-rp-1', 'user': 'root',...

tests: relax check in AutomaticLeadershipBalancingTest

6

Relax the shard leader count check because leader balancer may not be able to achieve balanced counts due to interplay between topic-aware and total counts objectives (see https://github.com/redpanda-data/core-internal/issues/1282). Fixes https://github.com/redpanda-data/redpanda/issues/17150...

Shard placement table persistence

4

Add persistence to `shard_placement_table`. Now all shard placement updates are persisted in kvstore and restored at startup. Also implement migrating from old topic table-determined placement and update the stress test...

[v23.2.x] Fix some concurrent memory access problems in partition balancer

1

Backport of PR https://github.com/redpanda-data/redpanda/pull/18305

Node-local shard assignments for created/moved partitions

3

## Backports Required - [x] none - not a bug fix - [ ] none - this is a backport - [ ] none - issue does not exist in...