nebula-operator icon indicating copy to clipboard operation
nebula-operator copied to clipboard

after restart a single storage pod, lead not balanced

Open jinyingsunny opened this issue 1 year ago • 4 comments

after restart a single storage pod, operator do balance leader, but leader still not balanced.

image

check operator log, may be the action too early,from ready to restart storaged pod to do balance leader,only less than 500ms's interval. image

since we know, do balance leader may not archive the aim for once, may be we should do some check and repeat.

Your Environments (required)

operator镜像:reg.vesoft-inc.com/cloud-dev/nebula-operator:snap-1.35

How To Reproduce(required)

Steps to reproduce the behavior:

1. with 3 zones and each zone with 3 storaged;
2. create 2 space;
3. restart one storaged pod `kubectl -n nebula annotate sts nebulav-storaged nebula-graph.io/restart-ordinal="8"`
4. pay attention to operator log and check storaged leader distribution. eg: show hosts.

Expected behavior after restart , leader keep balanced

jinyingsunny avatar Feb 20 '24 03:02 jinyingsunny

补充:当3个zone,每个zone分别只有1个sotraged时,重启了一个storaged后,虽然在两个space中都完成了 balance leader,但最终结果,依旧是不均匀。 image

相关balance操作的日志: image

jinyingsunny avatar Feb 20 '24 06:02 jinyingsunny

checked with snap-1.37, still has the problem

jinyingsunny avatar Feb 26 '24 02:02 jinyingsunny

discues offline: remind user to do balance data by hand;later do optimize.

jinyingsunny avatar Mar 04 '24 11:03 jinyingsunny

@abby-cyber

jinyingsunny avatar Mar 04 '24 11:03 jinyingsunny

https://github.com/vesoft-inc/nebula-operator/pull/507

MegaByte875 avatar May 30 '24 02:05 MegaByte875