Is it possible to install microk8s on ubuntu core?

[lejsekt]

Hi, is it possible to install microk8s on rabsperry pi 4 with ubuntu core 20 for arm64?

sudo snap install microk8s --classic error: snap "microk8s" requires classic confinement which is only available on classic systems

Thank you

[balchua]

I think Ubuntu Core supports only strict confinement.

[lejsekt]

And is it planned to support Ubuntu Core in the future?

[ktsakalozos]

Indeed Ubuntu Core needs strict confinement. There is a branch where we got most of MicroK8s strictly confined [1]. @joedborg has worked on this.

@lejsekt what would you need a strictly confined MicroK8s for? We would like your input to make a case to work on it. Thanks.

[1] https://github.com/ubuntu/microk8s/tree/feature/jdb%2Fstrict

[lejsekt]

@ktsakalozos Well, my use case is the following. I'd like to setup a fleet of IoT devices and manage them through microk8s. Ubuntu Core seems like a good fit as a base OS for these devices. So it'd be great to be able to turn these Ubuntu Core devices into microk8s nodes.

[mikepurvis]

I'm interested as well, though I don't have a great motivation either other than looking for the smallest, quickest setup for rolling out a bunch of these nodes.

[dylancourt]

+1 for this! I work at a smart building company and we're looking to replace yocto os while also having a pathway to kubernetes on embedded devices at Edge sites

[balchua]

@ktsakalozos im adding this to the wishlist. 😊

[usrbinkat]

microk8s on top of ubuntu core would simply solve the hardest challenges that I face with deployment and lifecycle of assembly plant robots and low power edge compute at 5G tower installations. I could enumerate the layers of features that enhance the end to end lifecycle if needed but both together would be the compelling story that I'm really looking for.

Please let me know if I can help get visibility to the importance of this work.

[ncpe20012003]

Any progress? I have a need to run microk8s on UC 20.

[robert-schmuck]

Hi, I am also very interested in running mickrok8s on ubuntu core 20, is there plan when to implement this?

[ncpe20012003]

Checking in, any word on MicroK8s in UC 20?

[balchua]

@ncpe20012003 i know there's been plenty of work being put on this. @ktsakalozos maybe able to share more on this.

[ktsakalozos]

We are actively working on getting a strictly confined snap. There is a branch where you could follow our progress https://github.com/ubuntu/microk8s/tree/feature/jdb%2Fstrict . The build for now go in the latest/edge/strict channel. You could give it a try with:

sudo snap install microk8s --channel=latest/edge/strict --devmode

[Christoph02]

Also waiting on microk8s for ubuntu core. Tried on proxmox and ran in same issue. Would be great to have this very cool snap on ubuntu core, too. currently not working with --channel=latest/edge/strict even not in devmode.

Kind regards, Christoph

[Ch4s3r]

@ktsakalozos There was a typo in your command in mcirok8s. Works awesome otherwise on ubuntu core :)

sudo snap install microk8s --channel=latest/edge/strict --devmode

[ktsakalozos]

We would appreciate if you could run in non-devmode (without the --devmode) and provide us with some feedback on what may be broken. An inspection report should be enough (microk8s inspect) . Thank you.

[Ch4s3r]

There was an error on my side, I only tested it on a VM and expected it to also work on a PI. So it's currently not working on the Pi 4 directly (with or without devmode). Error message that I get on both from sudo microk8s inspect: microk8s is not running. Use microk8s inspect for a deeper inspection.

But anyway here are the two reports if they still help: devmode: inspection-report-20211004_213459.tar.gz non-devmode: inspection-report-20211004_205641.tar.gz

Both setup on a fresh ubuntu core 20 arm64 distro on a Pi 4.

[ktsakalozos]

@Ch4s3r in the logs I see this error:

Oct 04 21:34:40 ubuntu microk8s.daemon-kubelite[7068]: E1004 21:34:40.630574    7068 kubelet.go:1423] "Failed to start ContainerManager" err="system validation failed - Following Cgroup subsystem not mounted: [memory]"

Have you enabled cgroups as described in https://microk8s.io/docs/install-alternatives#heading--arm ?

[Ch4s3r]

I did but maybe in the wrong place. Because I could not find the firmware folder but there was another cmdline.txt somewhere in /writable/.../system-data/... Will have to recheck this in the evening.

[Ch4s3r]

Tried but could not append the kernel command line arguments on Ubuntu Core 20. There is no /boot/firmware folder to write them to and there is no possibility to create this folder as the system is read-only. Any suggestions how to do this?

[Ch4s3r]

Had to add cgroup_enable=memory cgroup_memory=1 to /ubuntu-seed/cmdline of the sd card.

Now it's successfully shown that cgroups are enabled:

> grep memory /proc/cgroups
memory  6       269     1

But still getting this:

> sudo microk8s status
microk8s is not running. Use microk8s inspect for a deeper inspection.

Here's another log (with devmode): inspection-report-20211006_201819.tar.gz

[Ch4s3r]

Somehow it started working after invoking sudo microk8s status multiple times or maybe it was also a reboot in-between.

Here the log of a working non-devmode microk8s on Ubuntu Core 20: inspection-report-20211006_210501.tar.gz 🥳

It also shows this on the status which is a bit worrying:

Fatal: can't open lock file /run/xtables.lock: Permission denied
/snap/microk8s/2556/inspect.sh: line 187: /bin/mount: Permission denied
/snap/microk8s/2556/inspect.sh: line 188: /bin/mount: Permission denied
WARNING:  The memory cgroup is not enabled.
The cluster may not be functioning properly. Please ensure cgroups are enabled

But again i verified that the memory cgroup is indeed enabled:

> grep memory /proc/cgroups
memory  6       280     1

[Christoph02]

Thank you Konstantinos (https://github.com/ktsakalozos)

MicroK8s v1.25 released!

This is a real milestone: Strict confinement goes into general availability

It worked for me with Ubuntu Core on Raspberry 4.

snap install microk8s --channel=1.25-strict/stable
microk8s (1.25-strict/stable) v1.25.0 from Canonical✓ installed

The IoT experience with Ubuntu Core and MicroK8s can continue, now ...

Thanks a lot and best regards, Christoph

[usrbinkat]

This is exciting!

[joedborg]

So happy to see this go GA! Well done everyone.

[edocalia]

Thanks everyone, this is one major step beyond!

[Duc]

Hi, I successfully installed and run MicroK8s v1.26.0 revision 4393 on Ubuntu Core 22. I'm in a dead end now regarding NFS volume, It's seems that Ubuntu Core is not able to mount NFS share. Am I right ?

[ktsakalozos]

@Duc I asked over at https://forum.snapcraft.io/t/mount-nfs-share-on-ubuntu-core/33332 as this might be a topic for snaps on core in general.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.