k8s-rdma-shared-dev-plugin icon indicating copy to clipboard operation
k8s-rdma-shared-dev-plugin copied to clipboard

Did not detect devices

Open aisfx opened this issue 7 months ago • 1 comments

error:

ib_write_bw Did not detect devices If device exists, check if driver is up Unable to find the Infiniband/RoCE device

kubectl apply -f test.yaml apiVersion: v1 kind: Pod metadata: name: pod1 annotations: k8s.v1.cni.cncf.io/networks: ens2np0,ens3np0 spec: restartPolicy: OnFailure containers:

  • image: mellanox/rping-test name: mofed-test-ctr securityContext: capabilities: add: [ "IPC_LOCK" ] resources: limits: rdma/hca: 1 requests: rdma/hca: 1 command:
    • sh
    • -c
    • | ls -l /dev/infiniband /sys/class/infiniband /sys/class/net sleep 1000000

infiniband input info:

/dev/infiniband: │ │ total 0 │ │ crw-rw-rw- 1 root root 10, 121 Apr 6 09:26 rdma_cm │ │ crw------- 1 root root 231, 0 Apr 6 09:26 umad0 │ │ crw------- 1 root root 231, 1 Apr 6 09:26 umad1 │ │ crw-rw-rw- 1 root root 231, 192 Apr 6 09:26 uverbs0 │ │ crw-rw-rw- 1 root root 231, 193 Apr 6 09:26 uverbs1 │ │ │ │ /sys/class/infiniband: │ │ total 0 │ │ lrwxrwxrwx 1 root root 0 Apr 6 09:26 mlx5_0 -> ../../devices/pci0000:03/0000:03:01.0/0000:04:00.0/0000:05:0c.0/0000:0f:00.0/0000:10:10.0/0000:1 │ │ lrwxrwxrwx 1 root root 0 Apr 6 09:26 mlx5_1 -> ../../devices/pci0000:2d/0000:2d:01.0/0000:2e:00.0/0000:2f:00.0/0000:30:00.0/0000:31:10.0/0000:3 │ │ lrwxrwxrwx 1 root root 0 Apr 6 09:26 mlx5_2 -> ../../devices/pci0000:56/0000:56:01.0/0000:57:00.0/infiniband/mlx5_2 │ │ │ │ /sys/class/net: │ │ total 0 │ │ lrwxrwxrwx 1 root root 0 Apr 6 09:26 eth0 -> ../../devices/virtual/net/eth0 │ │ lrwxrwxrwx 1 root root 0 Apr 6 09:26 lo -> ../../devices/virtual/net/lo │ │ lrwxrwxrwx 1 root root 0 Apr 6 09:26 net1 -> ../../devices/virtual/net/net1 │ │ lrwxrwxrwx 1 root root 0 Apr 6 09:26 net2 -> ../../devices/virtual/net/net2 │ │ lrwxrwxrwx 1 root root 0 Apr 6 09:26 tunl0 -> ../../devices/virtual/net/tunl0 │ │

aisfx avatar Apr 06 '25 09:04 aisfx

os: ubuntu:22.04 ibdrivers: MLNX_OFED_LINUX-24.10-2.1.8.0-ubuntu22.04-x86_64 instate :rdma link 0/1: mlx5_0/1: state ACTIVE physical_state LINK_UP 1/1: mlx5_1/1: state ACTIVE physical_state LINK_UP

aisfx avatar Apr 06 '25 09:04 aisfx