-pthread not set in cross-compiled python gcc flags

[virtuald]

Problem

Without -pthread set when compiling, bad things happen, such as this:

ImportError: /usr/local/lib/python3.8/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-38-arm-linux-gnueabi.so: undefined symbol: pthread_atfork

On my native python (Fedora 32, Python 3.8):

>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'gcc -pthread',
 'CXX': 'g++ -pthread',
 'LDSHARED': 'gcc -pthread -shared -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now  '
             '-g  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now  -g'}

However in the cross-python:

>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'arm-frc2020-linux-gnueabi-gcc',
 'CXX': 'arm-frc2020-linux-gnueabi-c++',
 'LDSHARED': 'arm-frc2020-linux-gnueabi-gcc -shared'}

Potential fix

Unfortunately, python's configure script runs a program to determine if -pthread is needed. However, there's a partial fix for this. If one adds ac_cv_pthread_is_default=no ac_cv_pthread=yes ac_cv_cxx_thread=yes to the cross ./configure argument, you get:

>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'arm-frc2020-linux-gnueabi-gcc -pthread',
 'CXX': 'arm-frc2020-linux-gnueabi-c++',
 'LDSHARED': 'arm-frc2020-linux-gnueabi-gcc -pthread -shared'}

It seems like there isn't a way to override the CXX version without changing configure.ac (unless you have a good idea), filed an issue against CPython @ https://bugs.python.org/issue41916

Action for crossenv

Probably all/some of this discussion should be integrated into the cross-compilation documentation? Not sure how you'd like that to be done.

[benfogle]

This sounds like a toolchain/cpython issue more than a crossenv issue?

When creating the cross environment, would adding --cxx='arm-frc2020-linux-gnueabi-c++ -pthread' be a workaround? If that works then that should be better documented.

[virtuald]

Sorry for the delayed response! I haven't tried that workaround.

Instead, I ended up patching configure.ac/configure and that seems to have fixed the issue (PR @ https://github.com/python/cpython/pull/22525). With that patch applied, setting the configure cache values ac_cv_pthread_is_default=no ac_cv_pthread=yes ac_cv_cxx_thread=yes fixes the issue.