gpushare-scheduler-extender
gpushare-scheduler-extender copied to clipboard
ALIYUN_COM_GPU_MEM_IDX in the annotation is different than ALIYUN_COM_GPU_MEM_IDX inside the pod
Annotations:
Annotations: ALIYUN_COM_GPU_MEM_ASSIGNED: true
ALIYUN_COM_GPU_MEM_ASSUME_TIME: 1692105746106628538
ALIYUN_COM_GPU_MEM_DEV: 11
ALIYUN_COM_GPU_MEM_IDX: 4
ALIYUN_COM_GPU_MEM_POD: 2
Env:
ALIYUN_COM_GPU_MEM_DEV=11
ALIYUN_COM_GPU_MEM_IDX=3
ALIYUN_COM_GPU_MEM_POD=2
ALIYUN_COM_GPU_MEM_CONTAINER=2
The device:
NVIDIA_VISIBLE_DEVICES=GPU-280dd117-09e1-2e8c-25e3-52fdfac9527f
is indeed the 3rd device so the annotation is wrong and the environment variable is correct.