molassembler icon indicating copy to clipboard operation
molassembler copied to clipboard

Python bindings cannot link dynamically against shared library

Open awvwgk opened this issue 2 years ago • 4 comments

The Python bindings statically link against an archive rather than the shared library even if -DBUILD_SHARED_LIBS=ON is set. Is there a way to reduce the redundancy in the Python bindings by just linking the already available shared library?

awvwgk avatar Jul 07 '22 18:07 awvwgk

Why is it redundant for the python bindings to be linked against a static molassembler library? I think this was chosen specifically to reduce the amount of files and code present in the python packages. Molassembler code not reachable from python should then not be present in the python-loadable SO.

jan-grimo avatar Aug 17 '22 14:08 jan-grimo

For the Readuct build I can split the build in two stages, building the project without Python bindings first and afterwards building the Python bindings against the previously compiled shared library. This allows to reduce build times and package size, specially when building for many Python versions (currently CPython 3.7, 3.8, 3.9, 3.10 and PyPy 3.8, 3.9).

I tried doing the same for molassembler, but here I usually run into a linking issue if I try to use the shared library for linking rather than the static one.

awvwgk avatar Aug 17 '22 14:08 awvwgk

What's different between readuct and molassembler that the same approach but with the static library variant isn't feasible?

jan-grimo avatar Aug 17 '22 14:08 jan-grimo

For readuct I'm rediscovering the installed library using the CMake config file, for molassembler I cannot do this because I'm building with -DBUILD_SHARED_LIBS=ON and only the shared library is installed, which cannot be used to link the Python bindings.

There are several possibilities to work around this, like install the static library locally and reuse it in the next build stage or make the static library installable and create an additional package output (scine-molassembler-static). The build can take quite a while (see https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=552845&view=results), which makes testing new changes somewhat time-consuming.

awvwgk avatar Aug 17 '22 14:08 awvwgk