gpushare-scheduler-extender icon indicating copy to clipboard operation
gpushare-scheduler-extender copied to clipboard

ALIYUN_COM_GPU_MEM_IDX in the annotation is different than ALIYUN_COM_GPU_MEM_IDX inside the pod

Open wokalski opened this issue 2 years ago • 0 comments

Annotations:

Annotations:      ALIYUN_COM_GPU_MEM_ASSIGNED: true
                  ALIYUN_COM_GPU_MEM_ASSUME_TIME: 1692105746106628538
                  ALIYUN_COM_GPU_MEM_DEV: 11
                  ALIYUN_COM_GPU_MEM_IDX: 4
                  ALIYUN_COM_GPU_MEM_POD: 2

Env:

ALIYUN_COM_GPU_MEM_DEV=11
ALIYUN_COM_GPU_MEM_IDX=3
ALIYUN_COM_GPU_MEM_POD=2
ALIYUN_COM_GPU_MEM_CONTAINER=2

The device:

NVIDIA_VISIBLE_DEVICES=GPU-280dd117-09e1-2e8c-25e3-52fdfac9527f

is indeed the 3rd device so the annotation is wrong and the environment variable is correct.

wokalski avatar Nov 28 '23 14:11 wokalski