Nitin Mathur

@louiseschmidtgen For node3 we disconnected the netwrok adapter for the VM. We did not perform any operations on first node and second node.

@louiseschmidtgen We tried a few options. We reduced the load on our systems, we are running just microk8s and 2 test pods ubuntu (reference: https://gist.github.com/lazypower/356747365cb80876b0b336e2b61b9c26) We are able to reproduce...

@cole-miller Juet recreated the issue on the setup. This time, I set the log level of dqlite to `2`. We are reverting the machines to different states, so can't execute...

@louiseschmidtgen any update or recommendations for us to try?

@louiseschmidtgen Thanks for providing the patch. Yes, we will install and test it. We should have an update for you by tomorrow.

@louiseschmidtgen We did our testing with the dev version and below our observations. - When a node is disconnected we are not observing the other 2 nodes going in NotReady...

@louiseschmidtgen We saw a new issue on the dev version. For one of our run when we disconnected the network microk8s looses HA. This is microk8s status of all the...

Node Inspect reports Node1: [node-1-inspection-report-20240814_110705.tar.gz](https://github.com/user-attachments/files/16618253/node-1-inspection-report-20240814_110705.tar.gz) Node2: [node-2-inspection-report-20240814_110713.tar.gz](https://github.com/user-attachments/files/16618258/node-2-inspection-report-20240814_110713.tar.gz) Node3: [node-3-inspection-report-20240814_110646.tar.gz](https://github.com/user-attachments/files/16618260/node-3-inspection-report-20240814_110646.tar.gz)

For the inspect report, we disconnected node 1. It being the dqlite leader cluster became unhealthy. What our observation is that the cluster should be running in HA mode somehow...

@louiseschmidtgen @berkayoz We tried to reproduce the HA disconnect on our side. We are also not able to reproduce it. We think somehow our VMware snapshot was corrupted as every...