Cannot use systemd cgroups in inner docker

[joeljeske]

I am able to run the container nestybox/ubuntu-focal-systemd-docker:latest in sysbox with the flag native.cgroupdriver=systemd passed to the inner-docker daemon. Unfortunately, if I re-create the container image from source using the Dockerfiles from https://github.com/nestybox/dockerfiles/blob/5b7ec2230af7fb65eb820277e8c408cfa68f79b7/ubuntu-focal-systemd-docker/Dockerfile then and pass the same flag to the inner docker daemon I get the following error when I launch inner containers:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration: mkdir /sys/fs/cgroup/misc/system.slice: permission denied: unknown.
ERRO[0001] error waiting for container: 

I expect some set of deb packages are no longer compatible, but I'm un-usure which (e.g. systemd, docker-ce, containerd)

Conversation: https://nestybox-support.slack.com/archives/CS7V68QMP/p1698693917681689

[joeljeske]

It appears that downgrading containderd.io to 1.6.20 fixes this problem, the problem is introduced in 1.6.21. I am not sure which commits in this release break this functionality.

[ctalledo]

Thanks @joeljeske for reporting and finding out that the containerd version makes a difference. It's likely a bug / missing feature in Sysbox, exposed by a change in containerd. I'll take a look soon since we must ensure the latest versions of containerd work properly.

Thanks again.