gpu-manager icon indicating copy to clipboard operation
gpu-manager copied to clipboard

The POD created using JupyterHub sets the GPU threshold. There is no effect

Open fangpochen opened this issue 3 years ago • 11 comments

image image image As is shown in the figure. My total GPU has 100 computing power and 31* 256MB of video memory, about 8G. I set the limit to only 5% of 5* 356MB of video memory. This is clearly beyond

fangpochen avatar Apr 28 '21 02:04 fangpochen

@mYmNeo @aisensiy

fangpochen avatar Apr 28 '21 02:04 fangpochen

是和镜像有关吗。我单独使用yaml 文件创建pod。使用的是腾讯的文档创建的pod menghe.tencentcloudcr.com/public/tensorflow-gputest:0.2 这个测试镜像能够做到限制显存。但是,我自己在jupyterhub 的自定义镜像,却无法限制显存。这个有关系吗

fangpochen avatar Apr 28 '21 02:04 fangpochen

Does it have to do with mirror images. I used the YAML file alone to create the POD. The use is Tencent's document created by POD Menghe.tencentcloudcr.com/public/tensorflow-gputest:0.2 this test image can limit memory. However, my own custom image in JupyterHub does nothing to limit video memory. Does that matter

fangpochen avatar Apr 28 '21 02:04 fangpochen

Please check if you have override the environment variable LD_LIBRARY_PATH

mYmNeo avatar Apr 28 '21 04:04 mYmNeo

Please check if you have override the environment variable LD_LIBRARY_PATH

LD_LIBRARY_PATH 这个配置我在哪里查看呢

fangpochen avatar Apr 28 '21 05:04 fangpochen

image image

fangpochen avatar Apr 28 '21 05:04 fangpochen

这是我的。配置,并没有找到LD_LIBRARY_PATH 这个配置

fangpochen avatar Apr 28 '21 05:04 fangpochen

image 这是我容器内部的 LD_LIBRARY_PATH

fangpochen avatar Apr 28 '21 06:04 fangpochen

我的环境是 k8s 1.14 docker 1.20 gpu rtx2080 8g

fangpochen avatar Apr 28 '21 07:04 fangpochen

这个需要看你程序启动的脚本是不是覆盖了环境变量

mYmNeo avatar May 13 '21 01:05 mYmNeo

@fangpochen 问题解决了吗请问

mikev4 avatar Dec 08 '22 07:12 mikev4