Add support k3OS + Sysbox

[felipecrs]

It would be very useful if there were a way to install sysbox on k3OS. k3OS uses the Ubuntu's LTS kernel, which is 5.4 currently.

However, it does not have apt or dpkg for installing packages.

It seems that k3OS uses runc instead of docker due to k3s.

[rodnymolina]

@felipecrs, if i'm understanding K3OS approach correctly, K3OS is only capable of launching containers through K3s. If that's the case, then, before we deal with this issue, we must first allow K3s to launch Sysbox PODs, which is something we don't quite support yet. So sounds like we may need to postpone this one till Sysbox PODs feature is implemented.

Let me know if i'm missing something here (K3OS is new to me).

[felipecrs]

Please tell me if any of the following points are wrong:

• sysbox only supports docker, not containerd
• there is how to use docker instead of containerd in k3s: https://rancher.com/docs/k3s/latest/en/advanced/#using-docker-as-the-container-runtime
• after setting up k3s to use docker instead of containerd, we would only need to setup docker to use sysbox as the runtime
• k3OS: how to install Docker and make underlying k3s use it instead? (not a sysbox problem, I'll probably open an issue there)
• k3OS: how to install sysbox? Is it possible?

[felipecrs]

I'm sorry about this, I'm now a bit confused about the technologies.

But what prevents pods to be spawned using sysbox?

[rodnymolina]

Not a problem at all Felipe, too many technologies and acronyms to keep track of in this space ;-) Please see below ...

• Sysbox is a low-level runtime, so it executes at the same level of runc. Sysbox interacts with higher-level runtimes such as containerd (over which docker relies). Fyi, our goal is to have Sysbox talking to all OCI-compliant high-level runtimes (i.e. podman, crio, etc).

• K8s/K3s can rely on 1) docker+containerd OR 2) just containerd (there are other approaches, but let's stick to these two for now). And you are suggesting to make use of 1) coz Sysbox is already able to handle docker containers. But it's not that simple, coz in K8s case we are dealing with PODs, not just individual containers.

The issue with PODs is that there are shared kernel namespaces among the various 'containers' that form a POD, and there is a tricky situation that we need to handle when dealing with unprivileged user-namespaces and shared network-namespaces. I'll create an issue for this one soon so that you can read through all the details if you're curious about it.

• Concerning this: "k3OS: how to install sysbox? Is it possible?" Mmm, there's no apt-get and no dpkg, so system is expected to be inmutable ... no installation possible other than through Helm channels. Can't think about any trivial approach to fix this, would need to think deeper about it.

Hope this makes sense, let me know otherwise.

[rodnymolina]

@felipecrs, this is the issue i talked to you about: #67.

[felipecrs]

Makes super sense, thank you so much for the explanations.

Regarding PODs, maybe my own situation can be a bit different since I want to use sysbox to spawn Docker in Docker Jenkins build agents (with my image jenkins-agent-dind), using the Kubernetes plugin for Jenkins. In that case, each pod will have a single container, which will be the build agent.

In any case, I subscribed to https://github.com/nestybox/sysbox/issues/64.

[felipecrs]

Just as a heads-up: I'm not waiting for this feature, and I'm not a potential user of it as well.