python-build-standalone
python-build-standalone copied to clipboard
Restore shared extension module and/or dependency libraries
This project was created in support of PyOxidizer. And a goal of PyOxidizer was to see if it was possible to achieve a single file executable Python application.
In order to achieve that goal, this project built highly statically linked Python distributions.
On Linux and macOS, extension modules are statically linked into libpython. And 3rd party library dependencies of extension modules (OpenSSL, SQLite, etc) are also statically linked into libpython.
This mostly just works. Until it doesn't.
For example, ELF symbols resolution prefers symbols in the global namespace over a local namespace. So if you load a .so providing symbol foo which is also provided by an external libpython.so, the libpython.so symbol is used. This leads to wonkiness like https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html#static-linking-of-libx11-incompatibility-with-pyqt-on-linux.
As the Python distributions in this project have grown a life outside of PyOxidizer, the opinionated choice of statically linking all extension modules and shared libraries is hard to justify. We're foregoing compatibility with aspects of the larger Python ecosystem.
So I'm proposing that we revert to distributing 3rd party libraries and extension modules as shared libraries.
I looked at this over the weekend. I was soft blocked after running into a bug with patchelf corrupting ELF binaries. I need to read up on how the GNU dynamic loader handles rpath and $ORIGIN before I continue with an alternate solution.