xenon icon indicating copy to clipboard operation
xenon copied to clipboard

[question] stop xenon running on the master

Open WLonion opened this issue 5 years ago • 6 comments

Hi, I had configure replication with xenon. When I do a test, something confused me.

When I stopped the xenon running on the master, the master was remove from the replication. For example, A is master, B and C are slave. xenon running on the master A was stopped, the replication became B(new master) to C(slave). A became other instance not in the replication. Why? There are no problems on the master A, in fact A was serviceable, only the xenon was stopped.

From the document「1.3 How Raft+ Works」 When the Master is not serviceable, Slave1 and Slave2 immediately start a new winner.

Can you make me understand it? Thank you.

WLonion avatar Jul 29 '19 09:07 WLonion

Hi,

xenon is a MySQL agent, where it employs the raft protocol for the leader election. If we stop the xenon of A, means that the leader(raft) is down, slave-B and slave-C can't hear from the leader, they will do a new round election.

Followers handle the leader's heartbeat: https://github.com/radondb/xenon/blob/master/src/raft/follower.go#L74

Leader broadcasts heartbeat to the Followers: https://github.com/radondb/xenon/blob/master/src/raft/leader.go#L122

BohuTANG avatar Jul 29 '19 12:07 BohuTANG

OK. Thank you for your reply.

In my test, I use consul to replace vip. when the new master elected, brain split occurs. old master and new master also can be found by the consul as read and write instance. Now I'm considering how to remove the old master from consul using "leader-start-command" and "leader-stop-command". It's not clear. Do you have any experiences share with me?

WLonion avatar Jul 31 '19 07:07 WLonion

In my situation, I think it would be better if xenon have variables such as 'xenon-start-command' and 'xenon-stop-command'. Because command "xenoncli raft trytoleader" can trigger "leader-start-command" and "leader-stop-command", this is that what I don't want. :smile:

WLonion avatar Jul 31 '19 10:07 WLonion

xenon has 4 states when running: LEADER/FOLLOWER/CANDIDATE/IDLE leader-start-command only for the leader state starts where we want it to do the commands, such as vip mount: https://github.com/radondb/xenon/blob/master/src/raft/leader.go#L412

we do not have xenon-start-command config, since they are no state to determine.

BohuTANG avatar Aug 01 '19 08:08 BohuTANG

Fot the question : how to remove the old master from consul using "leader-start-command" and "leader-stop-command"

I am not expert with consul, not sure whether we can get the latest register ip. From the xenon side, leader-stop-command:

remove ip command;

leader-start-command:

remove ip command;
register the ip command;

BohuTANG avatar Aug 01 '19 09:08 BohuTANG

we do not have xenon-start-command config, since they are no state to determine.

Not the 4 states.

I mean xenon process start or stop: https://github.com/radondb/xenon/blob/master/src/server/server.go#L163 https://github.com/radondb/xenon/blob/master/src/server/server.go#L189

Maybe I could consider using consul + vip. Thank you.

WLonion avatar Aug 01 '19 10:08 WLonion