microk8s is not running - on a 4 node Rasp Pi 3 B+ cluster

[steentottrup]

Summary

I've just install microk8s on 4 Rasp Pi 3 B+. They were installed with Ubuntu 22.04.4 64 bit server OS. The first 3 nodes are joined with the control plane etc. 4th node is just a worker. Node 1 boots off a USB HDD, other 3 are on SD cards. When I try to get status, all I get back is this text:

"microk8s is not running. Use microk8s inspect for a deeper inspection."

Trying to enable dns and storage etc. fails, here the output from 'microk8s enable dns':

Traceback (most recent call last): File "/snap/microk8s/6565/scripts/wrappers/enable.py", line 41, in enable(prog_name="microk8s enable") File "/snap/microk8s/6565/usr/lib/python3/dist-packages/click/core.py", line 764, in call return self.main(*args, **kwargs) File "/snap/microk8s/6565/usr/lib/python3/dist-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/snap/microk8s/6565/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/snap/microk8s/6565/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/snap/microk8s/6565/scripts/wrappers/enable.py", line 37, in enable xable("enable", addons) File "/snap/microk8s/6565/scripts/wrappers/common/utils.py", line 470, in xable protected_xable(action, addon_args) File "/snap/microk8s/6565/scripts/wrappers/common/utils.py", line 498, in protected_xable unprotected_xable(action, addon_args) File "/snap/microk8s/6565/scripts/wrappers/common/utils.py", line 514, in unprotected_xable enabled_addons_info, disabled_addons_info = get_status(available_addons_info, True) File "/snap/microk8s/6565/scripts/wrappers/common/utils.py", line 566, in get_status kube_output = kubectl_get("all,ingress") File "/snap/microk8s/6565/scripts/wrappers/common/utils.py", line 248, in kubectl_get return run(KUBECTL, "get", cmd, "--all-namespaces", die=False) File "/snap/microk8s/6565/scripts/wrappers/common/utils.py", line 69, in run result.check_returncode() File "/snap/microk8s/6565/usr/lib/python3.8/subprocess.py", line 448, in check_returncode raise CalledProcessError(self.returncode, self.args, self.stdout, subprocess.CalledProcessError: Command '('/snap/microk8s/6565/microk8s-kubectl.wrapper', 'get', 'all,ingress', '--all-namespaces')' returned non-zero exit status 1.

What Should Happen Instead?

No errors when I joined the nodes together, so I was hoping everything was working and I could start putting workloads/services in the cluster.

Reproduction Steps

I've installed the microk8s a few times now, first on 2 nodes, and latest on 4 to see if the number og nodes was the issue. Same thing every time. This is what I'm doing on the freshly install Ubuntu 22.04.4:

sudo apt update && sudo apt upgrade -y && sudo reboot

sudo nano /boot/firmware/cmdline.txt Adding 'cgroup_enable=memory cgroup_memory=1' to the file

sudo apt install linux-modules-extra-raspi

sudo snap install microk8s --classic

sudo usermod -a -G microk8s rasppi sudo chown -f -R rasppi ~/.kube

microk8s status --wait-ready

The last command seems to never return/end.

Introspection Report

Inspecting system Inspecting Certificates Inspecting services Service snap.microk8s.daemon-cluster-agent is running Service snap.microk8s.daemon-containerd is running Service snap.microk8s.daemon-kubelite is running Service snap.microk8s.daemon-k8s-dqlite is running Service snap.microk8s.daemon-apiserver-kicker is running Copy service arguments to the final report tarball Inspecting AppArmor configuration Gathering system information Copy processes list to the final report tarball Copy disk usage information to the final report tarball Copy memory usage information to the final report tarball Copy server uptime to the final report tarball Copy openSSL information to the final report tarball Copy snap list to the final report tarball Copy VM name (or none) to the final report tarball Copy current linux distribution to the final report tarball Copy asnycio usage and limits to the final report tarball Copy inotify max_user_instances and max_user_watches to the final report tarball Copy network configuration to the final report tarball Inspecting kubernetes cluster Inspect kubernetes cluster Inspecting dqlite Inspect dqlite

Building the report tarball Report tarball is at /var/snap/microk8s/6565/inspection-report-20240303_075153.tar.gz

inspection-report-20240303_075153.tar.gz