am0ral93

Good evening. Similar situation, does not work on version k8s 1.24. Writes 0 of 0 GPU. But on docker

> I have the following question. I use part of the machines on k8s version 1.23, one on 1.24. Earlier, you told me that docker no longer works on 1.24,...

sudo ctr run --rm -t docker.io/library/hello-world:latest hello-world Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps:...

> I don't know how containerd works with jhub. I tried downgrading to version 1.23.3 these 2 vpc's but they are still touchable on Docker GPU 0-0. I would not...

> Вы установили NVIDIA Container Toolkit и изменили конфигурацию докера для работы с Nvidia? В`/etc/docker/daemon.json` > > ``` > { > "default-runtime": "nvidia", > "runtimes": { > "nvidia": { >...

> can you show the output of `kubectl describe node` and `nvida-smi`? # kubectl describe node vpc11 Name: vpc11 Roles: Labels: beta.kubernetes.io/arch=amd64 beta.kubernetes.io/os=linux kubernetes.io/arch=amd64 kubernetes.io/hostname=vpc11 kubernetes.io/os=linux Annotations: csi.volume.kubernetes.io/nodeid: {"rook-ceph.cephfs.csi.ceph.com":"vpc11","rook-ceph.rbd.csi.ceph.com":"vpc11"} kubeadm.alpha.kubernetes.io/cri-socket:...

> and logs from nvidia device plugin? kube-test-container-584687df5f-vwz7k 1/1 Running 4 (24d ago) 85d nvidia-device-plugin-1655230975-22s26 1/1 Running 0 37h nvidia-device-plugin-1655230975-jnkgm 1/1 Running 0 37h nvidia-device-plugin-1655230975-kswvx 1/1 Running 0 37h nvidia-device-plugin-1655230975-wvvnp...

> How many nodes do you have? One of them is missing NVML library... try to find the solution to this 5 nodes, but for some reason only 4 are...

> How many nodes do you have? One of them is missing NVML library... try to find the solution to this On this occasion, I do not understand what to...