delocate
delocate copied to clipboard
delocate-wheel sets invalid relative paths to delocated dylibs (related to "purelib" subdirectory)
Describe the bug / To Reproduce My original wheel contains these files:
$ unzip -l dist/PyMyLib-0.1.17-cp39-cp39-macosx_12_0_arm64.whl
Archive: dist/PyMyLib-0.1.17-cp39-cp39-macosx_12_0_arm64.whl
Length Date Time Name
--------- ---------- ----- ----
4444243 03-15-2022 11:19 PyMyLib-0.1.17.data/purelib/PyMyLib/PyMyLib.cpython-39-darwin.so
105 03-15-2022 11:19 PyMyLib-0.1.17.data/purelib/PyMyLib/__init__.py
215 03-15-2022 11:19 PyMyLib-0.1.17.dist-info/METADATA
108 03-15-2022 11:19 PyMyLib-0.1.17.dist-info/WHEEL
18 03-15-2022 11:19 PyMyLib-0.1.17.dist-info/top_level.txt
619 03-15-2022 11:19 PyMyLib-0.1.17.dist-info/RECORD
--------- -------
4445308 6 files
It depends on Boost, ICU, and PCRE, which have been installed via Homebrew:
$ delocate-listdeps dist/PyMyLib-0.1.17-cp39-cp39-macosx_12_0_arm64.whl
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_atomic-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_chrono-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_date_time-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_filesystem-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_iostreams-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_log-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_random-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_regex-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_serialization-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_system-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_thread-mt.dylib
/opt/homebrew/Cellar/boost/1.78.0_1/lib/libboost_unit_test_framework-mt.dylib
/opt/homebrew/Cellar/icu4c/70.1/lib/libicudata.70.1.dylib
/opt/homebrew/Cellar/icu4c/70.1/lib/libicui18n.70.1.dylib
/opt/homebrew/Cellar/icu4c/70.1/lib/libicuuc.70.1.dylib
/opt/homebrew/Cellar/pcre/8.45/lib/libpcre.1.dylib
I call delocate-wheel
like this:
$ delocate-wheel -w dist-patched -v dist/*whl
The dylibs are properly copied into the delocated wheel:
$ unzip -l dist-patched/PyMyLib-0.1.17-cp39-cp39-macosx_12_0_arm64.whl
Archive: dist-patched/PyMyLib-0.1.17-cp39-cp39-macosx_12_0_arm64.whl
Length Date Time Name
--------- ---------- ----- ----
1040912 03-15-2022 12:31 PyMyLib.dylibs/libboost_unit_test_framework-mt.dylib
88048 03-15-2022 12:31 PyMyLib.dylibs/libboost_chrono-mt.dylib
288224 03-15-2022 12:31 PyMyLib.dylibs/libpcre.1.dylib
227408 03-15-2022 12:31 PyMyLib.dylibs/libboost_filesystem-mt.dylib
180704 03-15-2022 12:31 PyMyLib.dylibs/libboost_thread-mt.dylib
35456 03-15-2022 12:31 PyMyLib.dylibs/libboost_system-mt.dylib
2780192 03-15-2022 12:31 PyMyLib.dylibs/libicui18n.70.1.dylib
89968 03-15-2022 12:31 PyMyLib.dylibs/libboost_atomic-mt.dylib
154208 03-15-2022 12:31 PyMyLib.dylibs/libboost_iostreams-mt.dylib
1748992 03-15-2022 12:31 PyMyLib.dylibs/libicuuc.70.1.dylib
29723472 03-15-2022 12:31 PyMyLib.dylibs/libicudata.70.1.dylib
1337616 03-15-2022 12:31 PyMyLib.dylibs/libboost_log-mt.dylib
84288 03-15-2022 12:31 PyMyLib.dylibs/libboost_random-mt.dylib
449360 03-15-2022 12:31 PyMyLib.dylibs/libboost_regex-mt.dylib
35536 03-15-2022 12:31 PyMyLib.dylibs/libboost_date_time-mt.dylib
634416 03-15-2022 12:31 PyMyLib.dylibs/libboost_serialization-mt.dylib
105 03-15-2022 11:19 PyMyLib-0.1.17.data/purelib/PyMyLib/__init__.py
4462352 03-15-2022 12:31 PyMyLib-0.1.17.data/purelib/PyMyLib/PyMyLib.cpython-39-darwin.so
2376 03-15-2022 12:31 PyMyLib-0.1.17.dist-info/RECORD
108 03-15-2022 11:19 PyMyLib-0.1.17.dist-info/WHEEL
18 03-15-2022 11:19 PyMyLib-0.1.17.dist-info/top_level.txt
215 03-15-2022 11:19 PyMyLib-0.1.17.dist-info/METADATA
--------- -------
43363974 22 files
The output of delocate-listdeps
looks reasonable at first sight:
$ delocate-listdeps dist-patched/PyMyLib-0.1.17-cp39-cp39-macosx_12_0_arm64.whl
PyMyLib.dylibs/libboost_atomic-mt.dylib
PyMyLib.dylibs/libboost_chrono-mt.dylib
PyMyLib.dylibs/libboost_date_time-mt.dylib
PyMyLib.dylibs/libboost_filesystem-mt.dylib
PyMyLib.dylibs/libboost_iostreams-mt.dylib
PyMyLib.dylibs/libboost_log-mt.dylib
PyMyLib.dylibs/libboost_random-mt.dylib
PyMyLib.dylibs/libboost_regex-mt.dylib
PyMyLib.dylibs/libboost_serialization-mt.dylib
PyMyLib.dylibs/libboost_system-mt.dylib
PyMyLib.dylibs/libboost_thread-mt.dylib
PyMyLib.dylibs/libboost_unit_test_framework-mt.dylib
PyMyLib.dylibs/libicudata.70.1.dylib
PyMyLib.dylibs/libicui18n.70.1.dylib
PyMyLib.dylibs/libicuuc.70.1.dylib
PyMyLib.dylibs/libpcre.1.dylib
However, otool -L
shows me these relative paths:
$ cd dist-patched/
dist-patched$ unzip PyMyLib-0.1.17-cp39-cp39-macosx_12_0_arm64.whl
dist-patched$ otool -L PyMyLib-0.1.17.data/purelib/PyMyLib/PyMyLib.cpython-39-darwin.so
PyMyLib-0.1.17.data/purelib/PyMyLib/PyMyLib.cpython-39-darwin.so:
@loader_path/../../../PyMyLib.dylibs/libboost_date_time-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_iostreams-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_random-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_serialization-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_system-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_unit_test_framework-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_log-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libicudata.70.1.dylib (compatibility version 70.0.0, current version 70.1.0)
@loader_path/../../../PyMyLib.dylibs/libicui18n.70.1.dylib (compatibility version 70.0.0, current version 70.1.0)
@loader_path/../../../PyMyLib.dylibs/libicuuc.70.1.dylib (compatibility version 70.0.0, current version 70.1.0)
@loader_path/../../../PyMyLib.dylibs/libpcre.1.dylib (compatibility version 4.0.0, current version 4.13.0)
@loader_path/../../../PyMyLib.dylibs/libboost_chrono-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_regex-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_filesystem-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_thread-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
@loader_path/../../../PyMyLib.dylibs/libboost_atomic-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1200.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)
Note all the relative paths @loader_path/../../../PyMyLib.dylibs
. While these match the files as they are placed inside the wheel archive, after pip install
these relative paths don't point to the dylibs under site-packages:
dist-patched$ pip install -U --force-reinstall PyMyLib-0.1.17-cp39-cp39-macosx_12_0_arm64.whl
# in my virtual environment:
.venv-devel-macos-arm64-tf26/lib/python3.9/site-packages$ ls PyMyLib*
PyMyLib:
PyMyLib.cpython-39-darwin.so __init__.py __pycache__
PyMyLib-0.1.17.dist-info:
INSTALLER METADATA RECORD REQUESTED WHEEL direct_url.json top_level.txt
PyMyLib.dylibs:
libboost_atomic-mt.dylib libboost_iostreams-mt.dylib libboost_serialization-mt.dylib libicudata.70.1.dylib
libboost_chrono-mt.dylib libboost_log-mt.dylib libboost_system-mt.dylib libicui18n.70.1.dylib
libboost_date_time-mt.dylib libboost_random-mt.dylib libboost_thread-mt.dylib libicuuc.70.1.dylib
libboost_filesystem-mt.dylib libboost_regex-mt.dylib libboost_unit_test_framework-mt.dylib libpcre.1.dylib
Note how PyMyLib.cpython-39-darwin.so
and __init__.py
have been installed directly to PyMyLib/
, not to PyMyLib-0.1.17.data/purelib/PyMyLib/
(as it looks in the wheel archive).
Consequently, I get import errors because the dynamic libraries are not found:
$ python -c 'import PyMyLib'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "...../.venv-devel-macos-arm64-tf26/lib/python3.9/site-packages/PyMyLib/__init__.py", line 1, in <module>
from .PyMyLib import PyMyLib as MyLib
ImportError: dlopen(...../.venv-devel-macos-arm64-tf26/lib/python3.9/site-packages/PyMyLib/PyMyLib.cpython-39-darwin.so, 0x0002): Library not loaded: @loader_path/../../../PyMyLib.dylibs/libboost_date_time-mt.dylib
Referenced from: ...../.venv-devel-macos-arm64-tf26/lib/python3.9/site-packages/PyMyLib/PyMyLib.cpython-39-darwin.so
Reason: tried: '...../.venv-devel-macos-arm64-tf26/lib/python3.9/site-packages/PyMyLib/../../../PyMyLib.dylibs/libboost_date_time-mt.dylib' (no such file), '/usr/local/lib/libboost_date_time-mt.dylib' (no such file), '/usr/lib/libboost_date_time-mt.dylib' (no such file)
Expected behavior
delocate-wheel
should set relative paths to dylibs in such a way that they agree with their ultimate location after pip install
.
Wheels used
Unfortunately not open source. But the __init__.py
contains just these lines:
from .PyMyLib import PyMyLib as MyLib
if __name__ == "__main__":
pass
Platform (please complete the following information):
- OS version: macOS 12.2.1 (Apple Silicon)
- Delocate version: 0.10.2
$ pip --version
pip 22.0.4 from ...../.venv-devel-macos-arm64-tf26/lib/python3.9/site-packages/pip (python 3.9)
Additional context
I actually don't know why our (cmake-based) build process produces wheels with these subdirectories PyMyLib-0.1.17.data/purelib
. Pointers that could help me simplify that would also be appreciated. 🙂