python-pillow
Fix cross-compiling by searching the right lib and include directories
Fix cross-compiling by searching the right lib and include directories
We were previously searching the {sys.prefix}/lib and {sys.prefix}/include directories unconditionally. This is problematic when cross-compiling, as it does not take account of any sysroot where alternative libraries and headers are located. Adding -I/usr/include causes the build to explode, at least when cross-compiling from 64-bit to 32-bit.
Python does not officially support cross-compiling, but Gentoo achieves this by modifying the sysconfig variables like LIBDIR and INCLUDEDIR with great results.
Assuming "lib" is bad. 64-bit Linux systems often use lib64, putting 32-bit libraries under lib. You cannot assume that either though, as pure 64-bit Linux systems may just use lib instead. Things get even stranger on RISC-V.
The value of sys.prefix changes when using a virtualenv. Dependencies may be installed here, so it does make sense to continue supporting this case, even if it is incompatible with cross-compiling. Unlike regular environments, "lib" is generally used for libraries, although a lib64 symlink may also be present.
I slept on it and changed my mind about virtualenvs. I think dependencies may be installed there, so I have continued to support that case, even though it is incompatible with cross-compiling.
I don't suppose it would be possible for you to create a Dockerfile that demonstrates the need for this?
I did try very hard, but it's difficult to demonstrate on other distros I'm familiar with. I could use Gentoo, but then it might just look like Gentoo is doing something weird. Debian and friends avoid the issue with their multilib approach to cross-compiling. Fedora doesn't have a formal cross-compile mechanism, but you can usually make it work by setting a few flags. Unfortunately, it has some quirks that make it fall over in this case, and while I could work around them, I think that would just distract from the issue at hand. It's better to focus on the command that leads to the errors.
2024-03-03 14:30:32,691 root INFO armv7a-unknown-linux-gnueabihf-gcc --sysroot=/mnt/pi32 -Wsign-compare -DNDEBUG -O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard -DNDEBUG -fPIC -I/mnt/pi32/usr/include -I/mnt/pi32/usr/include/freetype2 -I/mnt/pi32/usr/include/harfbuzz -I/mnt/pi32/usr/include/glib-2.0 -I/mnt/pi32/usr/lib/glib-2.0/include -I/mnt/pi32/usr/include/fribidi -I/usr/include -I/mnt/pi32/usr/include/python3.11 -c src/_imagingmath.c -o build/temp.arm-linux-gnueabihf-cpython-311/src/_imagingmath.o
In file included from /mnt/pi32/usr/include/python3.11/Python.h:23,
from src/Tk/../libImaging/ImPlatform.h:10,
from src/Tk/../libImaging/Imaging.h:13,
from src/Tk/tkImaging.c:42:
/usr/include/stdlib.h:153:8: error: '_Float128' is not supported on this target
153 | extern _Float128 strtof128 (const char *__restrict __nptr,
| ^~~~~~~~~
/usr/include/stdlib.h:165:8: error: '_Float64x' is not supported on this target
165 | extern _Float64x strtof64x (const char *__restrict __nptr,
| ^~~~~~~~~
In file included from /mnt/pi32/usr/include/python3.11/Python.h:23,
from src/_imagingmorph.c:14:
/usr/include/stdlib.h:153:8: error: '_Float128' is not supported on this target
153 | extern _Float128 strtof128 (const char *__restrict __nptr,
| ^~~~~~~~~
/usr/include/stdlib.h:165:8: error: '_Float64x' is not supported on this target
165 | extern _Float64x strtof64x (const char *__restrict __nptr,
| ^~~~~~~~~
In file included from /mnt/pi32/usr/include/python3.11/pyport.h:218,
from /mnt/pi32/usr/include/python3.11/Python.h:38:
/usr/include/math.h:476:33: error: '_Float128' is not supported on this target
476 | # define _Mdouble_ _Float128
| ^~~~~~~~~
/usr/include/math.h:297:48: note: in definition of macro '__MATHDECL_1_IMPL'
297 | extern type __MATH_PRECNAME(function,suffix) args __THROW
| ^~~~
/usr/include/math.h:303:3: note: in expansion of macro '__MATHDECL_1'
303 | __MATHDECL_1(type, function, suffix, args)
| ^~~~~~~~~~~~
/usr/include/bits/mathcalls-helper-functions.h:20:1: note: in expansion of macro '__MATHDECL_ALIAS'
20 | __MATHDECL_ALIAS (int, __fpclassify,, (_Mdouble_ __value), fpclassify)
| ^~~~~~~~~~~~~~~~
/usr/include/bits/mathcalls-helper-functions.h:20:40: note: in expansion of macro '_Mdouble_'
20 | __MATHDECL_ALIAS (int, __fpclassify,, (_Mdouble_ __value), fpclassify)
| ^~~~~~~~~
error: command '/usr/bin/armv7a-unknown-linux-gnueabihf-gcc' failed with exit code 1
The above is truncated. It actually spews out pages and pages of this stuff. 64-bit x86_64 headers from /usr/include, particularly those under /usr/include/bits, massively break 32-bit targets like armv7a. Simply taking out -I/usr/include makes this work. You never need to add -I/usr/include because native toolchains will just look there anyway. I hope that makes it clearer.