Mixing system packages with python results in shared library errors

[bobvanderlinden]

Describe the bug

When using system packages from within a python application symbol lookup error may happen. This happens when the system package (like git) is linked against a different version of libc compared to the python version (from devenv) is linked against.

A specific case where this happens more often occurs when using a remote git repository as a requirement for your devenv-based python project. devenv calls pip install -r requirements.txt, which will run git. git is usually not defined inside devenv, thus the required libc is likely to be different from what is used in devenv. The python wrapper within devenv runs python with LD_LIBRARY_PATH=$DEVENV_PROFILE/lib, which will force ld to use libraries within $DEVENV_PROFILE/lib. The libc that resides in $DEVENV_PROFILE/lib is a different version than what git assumes to be ld-ed against.

This results in errors like the following:

Collecting git+https://github.com/pypa/sampleproject.git (from -r /nix/store/g41hqm03k2179v5pkyv889gyrgb370g0-requirements.txt (line 1))
  Cloning https://github.com/pypa/sampleproject.git to /tmp/nix-shell.07Ag0j/nix-shell.tKP4lT/pip-req-build-jidkerj5
  error: subprocess-exited-with-error
  
  × git version did not run successfully.
  │ exit code: 127
  ╰─> [2 lines of output]
      git: error while loading shared libraries: __vdso_gettimeofday: invalid mode for dlopen(): Invalid argument
      [end of output]

Potential solutions

• Adding git to packages in devenv will resolve the problem with git, but on my system I got the next binary that resulted in the a similar error (gh). Adding all personal tools is one way to get over this problem.
• Avoid using other binaries within the python application by not using git.
• Resetting LD_LIBRARY_PATH upon calling other binaries from within the wrapped devenv python.
• Make sure Python libraries are always linked against specific (working) versions of native libraries. This can be done using patchelf/autopatchelf. In addition, LD_LIBRARY_PATH should not be used so that ld won't be forced to use different libraries. See PR https://github.com/cachix/devenv/pull/840 as a poc.

To reproduce

Follow the instructions from the readme:

https://gist.github.com/bobvanderlinden/f31299079b67ddcc1b2c6029cf6569f6

Version

devenv 1.0.2 (x86_64-linux)

[shoskensMagics]

Also running into this when calling system commands from within my Python application. Current band-aid solution is to reset LD_LIBRARY_PATH before doing any system command.

[ddogfoodd]

@shoskensMagics you mean running unset LD_LIBRARY_PATH?