How to handle failover of master1?

[ErikXu]

I used sealos to setup an HA kubernetes cluster include master1, master2, master3, node1, node2. I run the sealos command in master1, so the installation files and docker registry is located in master1. One day, master1 is down. I want to add a new master4. How could I achieve this? Thanks!

[fanux]

1. the registry run on master0, you should use nerdctl to run it on other master, and copy /registry Dir to other node.
2. sealos add --masters <master4 ip> on master0, or copy ~/.sealos/ DIR to other master node to exec this command.

[ErikXu]

There is no master0 in my scenario. The registry runs on master1, but /registry is not found in master1. Instead, I found a registry dir in the path /var/lib/sealos/data/default/rootfs/registry.

Here is the structure of registry dir in master1:

[root@master1 rootfs]# tree /var/lib/sealos/data/default/rootfs/registry/ -L 5
/var/lib/sealos/data/default/rootfs/registry/
└── docker
    └── registry
        └── v2
            ├── blobs
            │   └── sha256
            └── repositories
                ├── calico
                ├── coredns
                ├── etcd
                ├── kube-apiserver
                ├── kube-controller-manager
                ├── kube-proxy
                ├── kube-scheduler
                ├── labring
                ├── pause
                └── tigera

16 directories, 0 files

I should copy /var/lib/sealos/data/default/rootfs/registry to another master node, right?

BTW, What's the correct command to launch the registry using nerdctl?

I cannot find the command from the doc https://github.com/labring/sealos/blob/main/docs/4.0/docs/design/private-registry.md or ps command:

[root@master1 rootfs]# ps -ef | grep nerdctl
root       8890   8881  0 Aug17 ?        00:00:09 /usr/bin/nerdctl _NERDCTL_INTERNAL_LOGGING /var/lib/nerdctl/1935db59
root     134215   2010  0 17:32 pts/0    00:00:00 grep --color=auto nerdctl

[fanux]

The registry should mount /var/lib/registry

[cuisongliu]

copy /var/lib/sealos/data/default/rootfs/registry to another master node and ln -s /var/lib/sealos/data/default/rootfs/registry /var/lib/registry

[fanux]

And edit each node /etc/hosts ， Resolve the correct hub address

[ErikXu]

Thanks all gays. Last question, what is the exact command to start the registry in another master node using nerdctl?

[fanux]

nerdctl run -d -p 5000:5000 -v /var/lib/registry:/var/lib/registry registry:2

https://github.com/labring/cluster-image/blob/main/containerd/scripts/init-registry.sh

This is our start script.

[ErikXu]

Thanks all. I will try and feed you back ASAP.