Segmentation fault with python embedded by conda on macOS

[olivier-roussel]

When compiling SOFA + SofaPython3 within a conda environment, so against a python that is provided by a conda package, trying to import a Sofa python package then leads to a seg fault. The bug only appears in macOS. The bug also appear on a ARM macOS by Etienne who initially spotted the bug. The bug does not occur when linking against a self-compiled (similar version) of python. Digging a bit, this looks similar to https://github.com/pybind/pybind11/issues/3907. However, testing some minimalist example of a pybind module with a conda python does not reproduce the bug. So this should not be limited to conda or pybind.

The bug seems to be due to the fact that the python module links with a libpython library, which for some reason I don't get causes a segv with python versions that don't link with a python library (such as the one shipped with conda). Here the linkage information for python v3.10 shipped with conda (failing case on macOS):

/Users/ci/miniforge3/envs/sofa-dev-3/bin/python3.10:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.106.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)

And linkage info for python v3.10 compiled from source (works fine even if the python module also links with libpython):

/Users/ci/miniforge3/envs/sofa-dev-3/../sofa-dev/bin/python3.10:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1675.129.0)
	/Users/ci/miniforge3/envs/sofa-dev/lib/libpython3.10.dylib (compatibility version 3.10.0, current version 3.10.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

Linkage info for a faulty Sofa python bindings module, here Core (note the presence of libpython):

lib/python3.10/site-packages/Sofa/Core.cpython-310-darwin.so:
	@rpath/Core.cpython-310-darwin.so (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libSofa.Component.Collision.Response.Contact.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Visual.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofaPython3_d.1.0.dylib (compatibility version 0.0.0, current version 1.0.0)
	@rpath/libSofa.Component.Collision.Response.Mapper.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Collision.Geometry.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Mapping.Linear.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Mapping.NonLinear.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.StateContainer.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Constraint.Lagrangian.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Constraint.Lagrangian.Model.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Constraint.Lagrangian.Correction.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Mass.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Topology.Container.Dynamic.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.LinearSolver.Iterative.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.LinearSystem.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.ODESolver.Backward.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Constraint.Lagrangian.Solver.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.SolidMechanics.Spring.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Topology.Container.Grid.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Topology.Container.Constant.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libpython3.10.dylib (compatibility version 3.10.0, current version 3.10.0)
	@rpath/libSofa.Simulation.Graph.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Simulation.Common.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Simulation.Core.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Core.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.DefaultType.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.LinearAlgebra.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Helper.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Topology.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Geometry.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Type.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Compat.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Component.Compat.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	@rpath/libSofa.Config.23.12.99.dylib (compatibility version 0.0.0, current version 23.12.99)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 902.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

The segv disappear when the link with libpython is removed in Sofa python modules. This is propagated from several sources in SofaPython3 cmake files. A PR will follow to fix this.

This does not seem to affect other OS (at least Linux). I guess this is due to the different way dynamic libraries or modules are handled between macOS and Linux.

---

Steps to reproduce:

python
>>> import Sofa.Core

which outputs

---------------------------------------
Checking SOFA_ROOT and SOFAPYTHON3_ROOT
Using environment variable SOFA_ROOT: /Users/ci/miniforge3/envs/sofa-dev-3
---------------------------------------
zsh: segmentation fault  python

---

Environment:

Python version: 3.10.13 install with conda from conda-forge Sofa version: 23.12 SofaPython3 version: b13cf03b macOS 10.15 Darwin kernel v19.4.0