Choosing a specific GPU device rendering in docker console? / VGL_DISPLAY for VirtualGL

[sbx126]

Hi,

Thanks for this great project. It helps a lot and we really appreciate it. We follow the instructions to run that docker container and it works. How to have GPU-based rendering in docker console?

However, when we try to choose a specific GPU device rendering in docker console, it seems not work.

NVIDIA_VISIBLE_DEVICES=2 vglrun /opt/VirtualGL/bin/glxspheres64 CUDA_VISIBLE_DEVICES=2 vglrun /opt/VirtualGL/bin/glxspheres64

We try some command above, but vglrun still run on the device 1. Could you give us some advice for that?
Thank you very much.

[ehfd]

Hi, this is because the default GPU for EGL is used in https://github.com/ehfd/docker-nvidia-egl-desktop/blob/main/entrypoint.sh#L33. I'll add an envvar for this for easy changes.

[ehfd]

@sbx126 You can override using export VGL_DISPLAY="/dev/dri/card0" or use vglrun -d /dev/dri/card0. Note that the specific device must be provisioned using docker --device=/dev/dri. More complicated with Kubernetes and keeping VGL_DISPLAY to default is recommended in this case.

[sbx126]

Background: Our compute clusters, all managed via Slurm, the /dev/dri/card[n] devices are either not all present or have restrictive permissions . We can only use Singularity instead of docker. We'll find out the similar method as docker --device=/dev/dri.

Thank you very much!

[ehfd]

@sbx126 Interesting, could you please try both https://github.com/ehfd/docker-nvidia-egl-desktop and https://github.com/ehfd/docker-nvidia-glx-desktop ? And also, VGL_DISPLAY is meant to be set as egl if in Singularity, as --device=/dev/dri is not required this way. Not advised to change.

[sbx126]

--device=/dev/dri and vglrun -d /dev/dri/card[n] works for the docker.

In Singularity, just --nv is OK, if have permissions. With restrictive permissions, we install virtualgl=3.0.2 from virtualgl PreReleases. Then we use vglrun -d egl[n], it works. https://github.com/VirtualGL/virtualgl/pull/207

Thank you very much for helping.

[ehfd]

I'll update to vgl 3.0.2 when the official release is out. Closing after an option to set this variable is implemented.

[ehfd]

Specifying VGL_DISPLAY via environment variables is now possible. Closing when 3.0.2 is added.

[ehfd]

Version 3.0.2 is here.