Not working with julia version 1.10.1 on linux systems

[Brusa99]

Affects: JuliaCall

Describe the bug

The new version 1.10.1 release broke the package juliacall on Linux systems. When importing the package from python a segmentation fault occurs.

How to reproduce:

1. Have julia version 1.10.1 installed in your system (or no version at all, since juliacall installs julia if not present).
2. create a clean conda environment with python (3.9 or superior tested).
3. run pip install juliacall
4. open python interactive session and run import juliacall

Note: this does not happen with version 1.10.0 and on Mac devices.

Your system

The bug is present both on manjaro linux (Manjaro Linux x86_64, kernel: 6.1.71-1-MANJARO) and on Ubuntu (Ubuntu 20.04.5 LTS x86_64, kernel 5.15.0-1054-azure).

I raised the issue also on the julia github

[bjodah]

I experienced this as well, and came here to look if this was reported already, I'm using a Debian (testing-branch) system, using Python 3.11 and Julia 1.10.1.

[PallHaraldsson]

Is this possibly about a specific Python version(s) such as (or how they are compiled "GCC 11.2.0"): Python 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] on linux

I can't reproduce on (my Linux Mint):

$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import juliacall
>>> 
>>> juliacall.Main.VERSION
Julia: v"1.10.1"

I skipped 2. in case it matters, not sure about a "clean conda environment", and why should it matter?

In case it helps anyone, I did not get the latest version with pip install juliacall I got 0.9.14, maybe a problem for others getting even older? [I'm not sure how to get latest mian bracnh, in case anyone can help, with or without --pre or version.] I don't know if pip vs pip3 is an issue, or running like this, made no difference, with that latest (non-main branch):

$ python3 -m pip install --pre juliacall==0.9.15
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: juliacall==0.9.15 in ./.local/lib/python3.10/site-packages (0.9.15)
Requirement already satisfied: juliapkg~=0.1.8 in ./.local/lib/python3.10/site-packages (from juliacall==0.9.15) (0.1.10)
Requirement already satisfied: semantic-version~=2.9 in ./.local/lib/python3.10/site-packages (from juliapkg~=0.1.8->juliacall==0.9.15) (2.10.0)

[cjdoris]

The issue over on Julia: https://github.com/JuliaLang/julia/issues/53363

[feefladder]

For people looking for a ~fix~ workaround:

import juliapkg
juliapkg.require_julia("=1.10.0")
juliapkg.resolve()
from juliacall import Main as jl
jl.println("hello world!")

[dpinol]

I can reproduce this error with default python 3.11.4 from ubuntu 23.04, but it works with pyenv python versions 3.11.6, 3.11.8 & 3.12.2

[cjdoris]

OK, as far as I can tell (based on comments by @topolarity) the issue occurs when Python was not built with the --enable-shared flag (which in particular enables -fPIC). The Python in conda-forge does not use this flag but the Python built by pyenv does use it, which seems to be consistent with peoples' experiences.

Fortunately this flag can be detected. Could everyone who's experienced this (@Brusa99, @bjodah, @feefladder, @dpinol, @LilithHafner) run the following command

python -c "import sysconfig; print(sysconfig.get_config_var('PY_ENABLE_SHARED'))"

on whichever versions of Python you have installed. Please report back the value (0 or 1) and whether that version of Python segfaults or not.

The hypothesis is that it works when it is 1 and segfaults when 0. Fortunately this at least gives us a way to detect the problem, and issue warnings or restrict to Julia <=1.10.0 appropriately.

[cjdoris]

https://github.com/conda-forge/python-feedstock/issues/222

Looks like pyjulia already experienced some issues with statically linked Python.

[Brusa99]

I ran the command and got 0. I'm running python version 3.10.13 and julia version 1.10.1 and it causes the segfaults.

[dpinol]

I ran the command from a new virtualenv and got a 1, but since I didn't personally create the original virtual env (which I had already deleted), I'm not fully sure that I'm reproducing the same configuration

[feefladder]

I got a 0. Indeed I switched to PythonCall from PyJulia exactly because PyJulia had problems with statically linked python. This is also an issue over at conda-forge I believe. to further check if python is statically linked, I use echo $(which python) && ldd $(which python):

echo $(which python) && ldd $(which python)
/home/user/micromamba/envs/ahf/bin/python
        linux-vdso.so.1 (0x00007fffe4b75000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x000074ff836c9000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x000074ff836c1000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x000074ff836b9000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000074ff835c9000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000074ff833b9000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x000074ff833b1000)
        /lib64/ld-linux-x86-64.so.2 (0x000074ff83d99000)

And we can see that it is statically linked, because there is no link to libpython.so

[LilithHafner]

I have never been able to reproduce this issue and still cannot.

x@fedora:~$ python -c "import sysconfig; print(sysconfig.get_config_var('PY_ENABLE_SHARED'))"
1

This is consistent with @cjdoris's hypothesis.

[topolarity]

Unfortunately, the build flags depend on the maintainer (not just the default build flags from Python).

On my system (Ubuntu), the Debian-provided python3 is OK and the Ubuntu-provided python3.11 is bad.

They are both statically-linked (but report PY_ENABLE_SHARED anyway - a bug?):

$ ldd $(which python3.11) | grep python
$ ldd $(which python3) | grep python
$ python3 -c "import sysconfig; print(sysconfig.get_config_var('PY_ENABLE_SHARED'))"
1
$ python3.11 -c "import sysconfig; print(sysconfig.get_config_var('PY_ENABLE_SHARED'))"
1

The direct test is to inspect the relocations in the binary:

$ readelf -rW $(which python3.11) | grep environ
0000000000a92620  000006a400000005 R_X86_64_COPY          0000000000a92620 __environ@GLIBC_2.2.5 + 0
$ readelf -rW $(which python3) | grep environ
0000000000562cb8  0000002900000006 R_X86_64_GLOB_DAT      0000000000000000 environ@GLIBC_2.2.5 + 0

The R_X86_64_COPY relocation causes the segfault

[topolarity]

Also as a heads up, there's a work-around being landed for 1.10.3: https://github.com/JuliaLang/julia/pull/53643

[MilesCranmer]

Linking related issue seen in PySR: https://github.com/MilesCranmer/PySR/issues/561

[cjdoris]

There's a new version of juliacall out which requires julia <=1.10.0, so if you upgrade juliacall things should work again. I'll allow 1.10.3 and 1.11 again later once those are out and shown to work OK.

[cjdoris]

I confirmed that JuliaCall works with 1.10.3 so I have made a release (JuliaCall 0.9.20) with the updated julia compat entries.