aibrix
aibrix copied to clipboard
MountVolume.SetUp failed for volume "kube-api-access-wfmh2" no space left on device
🐛 Describe the bug
Warning FailedMount 25s (x19 over 37m) kubelet (combined from similar events): MountVolume.SetUp failed for volume "kube-api-access-wfmh2" : write /var/lib/kubelet/pods/26af5e70-f9b9-4c12-a9de-dcf2d4e93689/volumes/kubernetes.io~projected/kube-api-access-wfmh2/..2025_02_18_00_11_37.99831085/namespace: no space left on device
Warning FailedMount 2m1s (x11 over 22m) kubelet (combined from similar events): MountVolume.SetUp failed for volume "kube-api-access-tzrpb" : write /var/lib/kubelet/pods/a951d40b-518a-4b29-a3c9-36ef96b4f09c/volumes/kubernetes.io~projected/kube-api-access-tzrpb/..2025_02_17_23_18_11.2933250311/ca.crt: no space left on device
Filesystem Size Used Avail Use% Mounted on
/dev/root 2.0G 1.2G 775M 61% /
devtmpfs 24G 0 24G 0% /dev
tmpfs 24G 0 24G 0% /dev/shm
tmpfs 9.5G 6.2M 9.5G 1% /run
tmpfs 24G 248K 24G 1% /etc/machine-id
tmpfs 256K 0 256K 0% /mnt/disks
efivarfs 56K 24K 27K 48% /sys/firmware/efi/efivars
tmpfs 24G 0 24G 0% /tmp
overlayfs 24G 248K 24G 1% /etc
/dev/nvme0n1p8 11M 24K 11M 1% /usr/share/oem
/dev/nvme0n1p1 95G 57G 38G 60% /mnt/stateful_partition
tmpfs 2.0M 224K 1.8M 11% /var/lib/cloud
none 24G 38M 24G 1% /dev/kube_shm
tmpfs 24G 12K 24G 1% /var/lib/kubelet/pki
If has enough room but failed
Steps to Reproduce
latest version
kubectl apply -f samples/kvcache/kvcache.yaml
kubectl apply -f samples/kvcache/deployment.yaml
Expected behavior
it should not through any errors
Environment
nightly & GCP
look at this issue and notice the default CPU & Memory are wrong, it all use value 2 instead of 2Gi. Update the kubebuilder default value