kindbox in Ubuntu 20.04 on btrfs - Kubelet fatal error during startup

[matthewparkinsondes]

Attempting to use kindbox on top of Ubuntu 20.04 running a btrfs file system produces the following fatal error in the logs during the Kubelet startup process, resulting in an inability to successfully launch the cluster.

Sep 08 23:44:54 k8s-control-plane kubelet[13550]: W0908 23:44:54.468225   13550 fs.go:565] stat failed on /dev/sda5 with error: no such file or directory
Sep 08 23:44:54 k8s-control-plane kubelet[13550]: F0908 23:44:54.468257   13550 kubelet.go:1350] Failed to start ContainerManager failed to get rootfs info: failed to get device for dir "/var/lib/kubelet": could not find device with major: 0, minor: 51 in cached partitions map

cat /proc/mounts | grep /var/lib/kubelet

/dev/sda5 /var/lib/kubelet btrfs rw,noatime,compress-force=zstd:3,ssd,space_cache,subvolid=5,subvol=/ 0 0
/dev/sda5 /var/lib/kubelet btrfs rw,noatime,compress-force=zstd:3,ssd,space_cache,subvolid=5,subvol=/ 0 0

[ctalledo]

Thanks @matthewparkinsondes for filing this issue, we will take a look.

We typically use hosts with ext4 filesystems, so we had not bumped into this.

Per our side conversation, looks like disabling the LocalStorageCapacityIsolation feature on the (inner) kubelet provided a work-around (as explained here).

[almereyda]

The last link is unresolvable for me, but works when pointing directly to GitHub:

• https://github.com/kubernetes/kubernetes/issues/80633#issuecomment-550994513

[ctalledo]

The last link is unresolvable for me, but works when pointing directly to GitHub

Thanks @almereyda, I've fixed the link.