apptainer
Provision 32-bit NVIDIA driver libraries with --nv
Version of Apptainer
1.3.3
Expected behavior
Some applications (including Wine - for running Windows .exe in Linux) require 32-bit NVIDIA driver libraries. The NVIDIA container toolkit supports this option using NVIDIA_DRIVER_CAPABILITIES=compat32.
Actual behavior
Only 64-bit NVIDIA libraries are provisioned using --nv.
Therefore, it would benefit to add an (example) x86 directory within /.singularity.d/libs and provision 32-bit libraries.
What OS/distro are you running
Ubuntu 22.04.4
How did you install Apptainer
PPA
@ehfd What do you think about an option to set the library search path to go along with --nv? It may be difficult otherwise to discover the 32-bit libraries.
For that matter, does it work already now if you just set the LD_LIBRARY_PATH environment variable to search first in the 32-bit library directories? (I didn't try it or look in the code to see if it would likely work.)
It is likely that setting LD_LIBRARY_PATH will lead to having only the 32-bit libraries provisioned instead of both.
There are exactly same library names targetting different architectures such as /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0 and /usr/lib/i386-linux-gnu/libGLX_nvidia.so.0. The issue here is that because the library names are identical, only one or the other is available using your method, when both are frequently needed at the same time. Wine and many other graphical programs need both.
@DrDaveD
You need to run a mix of 32-bit and 64-bit applications in the same container? You can't run different containers for the different applications? I'm not sure that apptainer could accommodate mixing them, since it maps all of the gpu libraries into a single directory.
You need to run a mix of 32-bit and 64-bit applications in the same container? You can't run different containers for the different applications? I'm not sure that apptainer could accommodate mixing them, since it maps all of the gpu libraries into a single directory.
Graphical applications need both 32-bit and 64-bit NVIDIA libraries. This is the whole point. So what I suggested was to add /.singularity.d/libs and something like /.singularity.d/x86-libs and add both into LD_LIBRARY_PATH inside the container.
The NVIDIA container toolkit injects such libraries when NVIDIA_DRIVER_CAPABILITIES=compat32 is included. Thus, they push /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0 and /usr/lib/i386-linux-gnu/libGLX_nvidia.so.0 when NVIDIA_DRIVER_CAPABILITIES=compat32 is included.
Ok, that makes sense. I don't know who will implement it though. Would you submit a PR? I suggest calling the new directory /.singularity.d/libs32. We wouldn't want it enabled by default, so I guess there should be another command line option --nv32, or --gpu32 or --compat32 if it is to be more generic to work with the other gpu types.
By the way, I think --compat32 is the right option name since this may also hold for AMD ROCm.