replication-manager icon indicating copy to clipboard operation
replication-manager copied to clipboard

GET ERROR: Moving source lab_to_dr with empty password to preserve replication stream on new master

Open peterpan104 opened this issue 1 year ago • 11 comments

I have two clusters in repman

## cluster 1
Master01
Slave01

## cluster 2
Master02
Slave02

Master02 also acts as a slave to Master01 (using replication source lab_to_dr).

However, when Master02 failed, I received an error message indicating that Slave02 entered an empty password during the 'change master' process.

I was wondering replication manager supports this kind of architecture? I'm happy to provide more information if needed.

image

peterpan104 avatar Jul 08 '24 08:07 peterpan104

Yes it does as long as you declare your 2 clusters , each cluster it's source name and domain id add cluster-head = "cluster-haproxy-source1"


[cluster-haproxy-source1] 
title = "cluster-haproxy-source1"

db-servers-hosts = "127.0.0.1:3310,127.0.0.1:3311"
db-servers-prefered-master = "127.0.0.1:3310"
db-servers-credential = "root:mariadb"
db-servers-connect-timeout = 1
replication-source-name = "source1"
replication-credential = "root:mariadb"
force-slave-readonly = false
haproxy = true
haproxy-write-port = 3303
haproxy-read-port = 3302

[cluster-haproxy-source2]
title = "cluster-haproxy-source2"

cluster-head = "cluster-haproxy-source1"
db-servers-hosts = "127.0.0.1:3312,127.0.0.1:3313"
db-servers-prefered-master = "127.0.0.1:3312"
db-servers-credential = "root:mariadb"
db-servers-connect-timeout = 1
replication-source-name = "source2"
replication-credential = "root:mariadb"
force-slave-readonly = false
prov-db-domain = "2"

svaroqui avatar Jul 08 '24 08:07 svaroqui

Just we have not been able to make such topology work with replication filters so far

svaroqui avatar Jul 09 '24 09:07 svaroqui

Yes it does as long as you declare your 2 clusters , each cluster it's source name and domain id add cluster-head = "cluster-haproxy-source1"


[cluster-haproxy-source1] 
title = "cluster-haproxy-source1"

db-servers-hosts = "127.0.0.1:3310,127.0.0.1:3311"
db-servers-prefered-master = "127.0.0.1:3310"
db-servers-credential = "root:mariadb"
db-servers-connect-timeout = 1
replication-source-name = "source1"
replication-credential = "root:mariadb"
force-slave-readonly = false
haproxy = true
haproxy-write-port = 3303
haproxy-read-port = 3302

[cluster-haproxy-source2]
title = "cluster-haproxy-source2"

cluster-head = "cluster-haproxy-source1"
db-servers-hosts = "127.0.0.1:3312,127.0.0.1:3313"
db-servers-prefered-master = "127.0.0.1:3312"
db-servers-credential = "root:mariadb"
db-servers-connect-timeout = 1
replication-source-name = "source2"
replication-credential = "root:mariadb"
force-slave-readonly = false
prov-db-domain = "2"

Hi @svaroqui I don't want both clusters to use replication for the channel. Instead, I only want it used between master02 and master01 to ensure that master01 data can be written to master02.

Alternatively, is it possible to prevent slave02 from executing 'change master to channel lab_to_dr' when master02 failed? Or have other possible solutions?

It's just like:

## cluster 1
master01
	--slave01
	## cluser 2
	--master02 (using replication for channel lab_to_dr)
		--slave02

thanks for your reply.

peterpan104 avatar Jul 09 '24 10:07 peterpan104

multi source replication does not work the way you describe it. We should no build on unsafe topology , just consider one cluster channel01 made with master01<-slave01 and <-master02 on domain 01, and an other cluster channel02 master02 <- slave02 on domain 02, if this not mariadb please consider 8.3 the new tagged GTID , ( but we do not yet support tags )

svaroqui avatar Jul 09 '24 13:07 svaroqui

I think you could try not to name the source of the parent cluster and so the replication that glue the 2 clusters only naming the cluster02

svaroqui avatar Jul 09 '24 14:07 svaroqui

Could you tag the issue with mysql or mariadb ?

svaroqui avatar Jul 09 '24 14:07 svaroqui

Alternatively, is it possible to prevent slave02 from executing 'change master to channel lab_to_dr' when master02 failed?

I guess that we could fixe it as i already notice this issue

svaroqui avatar Jul 09 '24 14:07 svaroqui

Hi @svaroqui I'm testing with mariadb 10.4.

I tried the above solution

## cluster1 (not use 'replication-source-name' )
master01
	slave01
	master02


## cluster2 (replication-source-name = "test01")
master02
	slave02

But when master02 failed,slave02 is selected as master, but I still get the error Moving source with empty password to preserve replication stream on new master.

When master02 recovers, it will not become a slave of slave02.

image

I also hope that this problem will be solved. I'm happy to provide more information if needed. Thank you for your suggestions.

peterpan104 avatar Jul 10 '24 03:07 peterpan104

The problem is not about moving sources. your replication-manager clearly stated mysqlbinlog to rejoin using binary log. You have to specify using other method if you're not using binary logs to rejoin or specify your mysqlbinlog path installation

caffeinated92 avatar Jul 18 '24 08:07 caffeinated92

Hi @caffeinated92

Under the same parameter file, When master02 is not using replication-source, if master02 fails and recovers, it will automatically rejoin. Is there any difference between these scenarios?

When using both replication-source and regular replication simultaneously, is it possible to control replication-source so that it won't be transferred?

Thank you for your reply.

peterpan104 avatar Jul 22 '24 07:07 peterpan104

Can you provide us with the version your are using please also send your configs and logs in plain text, we have multiple paying user using that topology an not only for migration , but things have do be done correctly for that to work

svaroqui avatar Jul 22 '24 17:07 svaroqui