rabbitmq-server
rabbitmq-server copied to clipboard
khepri: `queue_leader_locator = balanced` doesn't behave as expected
Note: this issue is related to an unreleased feature
When Khepri is enabled, queue_leader_locator = balanced
doesn't work as expected (perhaps other strategies as well):
$ cat ~/balanced.conf
queue_leader_locator = balanced
$ bazel run start-cluster RABBITMQ_CONFIG_FILE=$HOME/balanced.conf
$ rabbitmqctl -n rabbit-0 enable_feature_flag raft_based_metadata_store_phase1
$ rabbitmqctl -n rabbit-0 import_definitions 5k-quorum-queues.json
Importing definitions in JSON from a file at "5k-quorum-queues.json" ...
# while the import is running, check how the leaders are distributed, for example:
$ rabbitmqctl --no-table-headers --quiet -n rabbit-0 list_queues leader | sort | uniq -c
744 rabbit-0@kura
# at the end, with 5k queues, I have a skewed distribution:
$ rabbitmqctl --no-table-headers --quiet -n rabbit-0 list_queues leader | sort | uniq -c
2327 rabbit-0@kura
1301 rabbit-1@kura
1372 rabbit-2@kura
The balanced
strategy behaves differently for the first 1000 queues and then just distributes queues randomly. Judging by the above, that first 1000 just goes to the first node (instead of being precisely distributed) and then the rest is indeed distributed randomly.
The issue is likely somewhere in: https://github.com/rabbitmq/rabbitmq-server/blob/main/deps/rabbit/src/rabbit_queue_location.erl