talos
talos copied to clipboard
siderolabs
Allow setting `cgroupRoot` in kubelet config
Feature Request
Allow setting cgroupRoot in kubelet config.
Description
When running Talos as a pod on a Talos node, both kubelets try to use the /kubepods cgroup which seems to cause problems with the instance in the pod. Both kubelets seem to try to destroy each other's cgroups, quickly killing the nested kubernetes container processes. This may be a bug in the kubelet if it's supposed to only manage pods it knows it created, but I think it may assume ownership over all cgroups under [cgroupRoot]/kubepods. I've only tested with kubelet v1.20.15 so far.
I'd like to discuss the pros/cons of allowing Talos users to set their own cgroupRoot since it's not allowed in Talos currently. I'm currently running a modified Talos image do work around the issue described above.
I run the Talos container as privileged, so maybe that is why it does. If there's a way to mask the host cgroup filesystem and use an entirely different cgroup mount in the pod i would pursue that instead of changing cgroupRoot. I will look more into that.
To be clear, I see both kubelets attempt to kill the other's processes under [cgroupRoot]/kubepods however it seems that only the host kubelet is successful.