WiP : CPython3 inclusion

[tlaurion]

trafficstars

This is still not working, and aimed at seeking help from the community into resolving packaging problems when crosscompiling python3 even today.

• Includes upstream fixes for cross compilation recognized problems (see patch in PR.)
• Builds against python 3.9.2 (debian's 11 current python. Yes, we depend on the host's python to compile destination's)

As of current state creating this draft, I am not able to force python using its own --with-libc="$(musl-cross_libraries)" which should point to musl-cross-make's libc.so. Python still attempts to link against host's libc, resulting in:

Traceback (most recent call last): File "/home/user/heads/build/x86/python-3.9.2/Lib/decimal.py", line 3, in from _decimal import * ImportError: /lib/x86_64-linux-gnu/libc.so: invalid ELF header

Recommendations are sparse, and were documented prior into #233 #689.

When those problems are resolved, this will: Fixes #233 #689

[tlaurion]

The builds fails at https://app.circleci.com/pipelines/github/tlaurion/heads/1205/workflows/4ad764c2-b1d4-4273-ba58-19cc322c44f0/jobs/10624?invite=true#step-103-3751, trying to link against host's libc instead of musl-cross-make's, even if specified properly in the module https://github.com/osresearch/heads/pull/1216/files#diff-51bef0474d1464907f19301711f3d641a3621d8d86d4f6072350c1bf290078b1R28.

--with-libc="$(musl-cross_libraries)" here is correctly expended to --with-libc=/root/project/crossgcc/x86/x86_64-linux-musl/lib/libc.so from environement variables passed to ./configure script here: https://github.com/osresearch/heads/pull/1216/files#diff-51bef0474d1464907f19301711f3d641a3621d8d86d4f6072350c1bf290078b1R13-R33, resulting in:

PKG_CONFIG=/bin/false READELF=/root/project/crossgcc/x86/bin/x86_64-linux-musl-readelf ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no have_openssl=no CFLAGS=-Os CC=/root/project/crossgcc/x86/bin/x86_64-linux-musl-gcc -D__MUSL__ -isystem /root/project/install/x86/include -L/root/project/install/x86/lib AR=/root/project/crossgcc/x86/bin/x86_64-linux-musl-ar LD=/root/project/crossgcc/x86/bin/x86_64-linux-musl-ld STRIP=/root/project/crossgcc/x86/bin/x86_64-linux-musl-strip NM=/root/project/crossgcc/x86/bin/x86_64-linux-musl-nm OBJCOPY=/root/project/crossgcc/x86/bin/x86_64-linux-musl-objcopy OBJDUMP=/root/project/crossgcc/x86/bin/x86_64-linux-musl-objdump PKG_CONFIG_PATH=/root/project/install/x86/lib/pkgconfig PKG_CONFIG_SYSROOT_DIR=/root/project/install/x86 ./configure --enable-optimizations --with-cxx-main=/root/project/crossgcc/x86/bin/x86_64-linux-musl-gcc --with-libc=/root/project/crossgcc/x86/x86_64-linux-musl/lib/libc.so --host x86_64-linux-musl --build x86_64 --disable-ipv6 --prefix= --exec-prefix=

Still, the following error happens since the wrong libc is picked (host vs musl-cross-make's). Look for "ImportError: /usr/lib/x86_64-linux-gnu/libc.so: invalid ELF header" below:

if test "xupgrade" != "xno"  ; then \
	case upgrade in \
		upgrade) ensurepip="--upgrade" ;; \
		install|*) ensurepip="" ;; \
	esac; \
	 _PYTHON_PROJECT_BASE=/root/project/build/x86/python-3.9.2 _PYTHON_HOST_PLATFORM=linux-x86_64 PYTHONPATH=/root/project/build/x86/python-3.9.2/build/lib.linux-x86_64-3.9:./Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__linux_x86_64-linux-gnu python3.9 -m ensurepip \
		$ensurepip --root=/root/project/install/x86/ ; \
fi
Traceback (most recent call last):
  File "/root/project/build/x86/python-3.9.2/Lib/decimal.py", line 3, in <module>
    from _decimal import *
ImportError: /usr/lib/x86_64-linux-gnu/libc.so: invalid ELF header

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "/root/project/build/x86/python-3.9.2/Lib/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/root/project/build/x86/python-3.9.2/Lib/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/root/project/build/x86/python-3.9.2/Lib/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl/pip/__main__.py", line 26, in <module>
  File "/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/cli/main.py", line 73, in main
  File "/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/commands/__init__.py", line 104, in create_command
  File "/root/project/build/x86/python-3.9.2/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/commands/install.py", line 17, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/cli/req_command.py", line 16, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/index/collector.py", line 15, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/distlib/compat.py", line 83, in <module>
  File "/root/project/build/x86/python-3.9.2/Lib/xmlrpc/client.py", line 135, in <module>
    from decimal import Decimal
  File "/root/project/build/x86/python-3.9.2/Lib/decimal.py", line 8, in <module>
    from _pydecimal import *
  File "/root/project/build/x86/python-3.9.2/Lib/_pydecimal.py", line 440, in <module>
    import contextvars
  File "/root/project/build/x86/python-3.9.2/Lib/contextvars.py", line 1, in <module>
    from _contextvars import Context, ContextVar, Token, copy_context
ImportError: /usr/lib/x86_64-linux-gnu/libc.so: invalid ELF header
Traceback (most recent call last):
  File "/root/project/build/x86/python-3.9.2/Lib/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/project/build/x86/python-3.9.2/Lib/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/project/build/x86/python-3.9.2/Lib/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/root/project/build/x86/python-3.9.2/Lib/ensurepip/__init__.py", line 210, in _main
    return _bootstrap(
  File "/root/project/build/x86/python-3.9.2/Lib/ensurepip/__init__.py", line 129, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/root/project/build/x86/python-3.9.2/Lib/ensurepip/__init__.py", line 38, in _run_pip
    return subprocess.run([sys.executable, "-c", code], check=True).returncode
  File "/root/project/build/x86/python-3.9.2/Lib/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/python3.9', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/tmp/tmp6ngaaq2s/setuptools-49.2.1-py3-none-any.whl\', \'/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/tmp/tmp6ngaaq2s\', \'--root\', \'/root/project/install/x86/\', \'--upgrade\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1.
make[1]: *** [Makefile:1255: install] Error 1
make[1]: Leaving directory '/root/project/build/x86/python-3.9.2'
tail /root/project/build/x86/log/python.log
-----
    from _contextvars import Context, ContextVar, Token, copy_context
ImportError: /usr/lib/x86_64-linux-gnu/libc.so: invalid ELF header
Traceback (most recent call last):
  File "/root/project/build/x86/python-3.9.2/Lib/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/project/build/x86/python-3.9.2/Lib/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/project/build/x86/python-3.9.2/Lib/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/root/project/build/x86/python-3.9.2/Lib/ensurepip/__init__.py", line 210, in _main
    return _bootstrap(
  File "/root/project/build/x86/python-3.9.2/Lib/ensurepip/__init__.py", line 129, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/root/project/build/x86/python-3.9.2/Lib/ensurepip/__init__.py", line 38, in _run_pip
    return subprocess.run([sys.executable, "-c", code], check=True).returncode
  File "/root/project/build/x86/python-3.9.2/Lib/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/python3.9', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/tmp/tmp6ngaaq2s/setuptools-49.2.1-py3-none-any.whl\', \'/tmp/tmp6ngaaq2s/pip-20.2.3-py2.py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/tmp/tmp6ngaaq2s\', \'--root\', \'/root/project/install/x86/\', \'--upgrade\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1.
make[1]: *** [Makefile:1255: install] Error 1
make[1]: Leaving directory '/root/project/build/x86/python-3.9.2'
make: *** [Makefile:408: /root/project/build/x86/python-3.9.2/.build] Error 1

Issue tracked under #689. This PR will stall until a fix is found, discussions can happen into referred tickets.