meson-python icon indicating copy to clipboard operation
meson-python copied to clipboard

RPATH goes missing when using both `install_rpath` and an internal shared library dependency

Open rgommers opened this issue 11 months ago • 7 comments

This is a case that probably hasn't been exercised before: a Python extension module that links against both a shared library that's part of the package and a shared library built in a subproject that's folded into the wheel by meson-python.

I ran into this in SciPy when adding a subproject. This worked as advertised, except for this scipy.special._ufuncs extension module, which depends on this libsf_error shared library as well as on the library in the subproject. What one sees then is that the installed wheel can no longer find libsf_error.so, because the RPATH entry for install_rpath: '$ORIGIN' goes missing.

With a default SciPy build (no subproject), the Library rpath|runpath entries on the _ufuncs target in the build and install directories looks like this (has $ORIGIN as it should):

$ readelf -d build/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so | rg "Library r"
 0x000000000000000f (RPATH)              Library rpath: [/home/rgommers/mambaforge/envs/scipy-dev-py312/lib:$ORIGIN/]

$ readelf -d build-install/lib/python3.12/site-packages/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so | rg "Library r"
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN:/home/rgommers/mambaforge/envs/scipy-dev-py312/lib]

On the branch with the subproject, we see this instead:

$ readelf -d build-whl/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so | rg Library
 0x000000000000001d (RUNPATH)            Library runpath: [/home/rgommers/code/pixi-dev-scipystack/scipy/.pixi/envs/openblas-src/lib:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs]

$ readelf -d site-packages/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so | rg "Library r"
 0x000000000000001d (RUNPATH)            Library runpath: [/home/rgommers/code/pixi-dev-scipystack/scipy/.pixi/envs/openblas-src/lib:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs]

The duplication of the .scipy.mesonpy.libs entries is a minor stylistic issue, we may be able to de-duplicate but it doesn't matter. What does matter is that $ORIGIN is no longer present.

This looks like a bug in the fix_rpath implementation at: https://github.com/mesonbuild/meson-python/blob/91d64d1d08aa2702ceaf1bde3edc53d69429100e/mesonpy/_rpath.py#L28-L36

The problem seems clear: all RPATH entries are cleared, and the ones that are added back all get libs_relative_path appended (i.e. to the subproject-relocated location). Any existing RPATHs within the package, like '$ORIGIN', will get lost.

rgommers avatar Dec 06 '24 20:12 rgommers