python
`__builtin_os_log_format_buffer_size` is missing for Catalina+GCC
Bug report
Bug description:
Build fails on macOS with gcc.
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/preconfig.o Python/preconfig.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pyarena.o Python/pyarena.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pyctype.o Python/pyctype.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pyfpe.o Python/pyfpe.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pyhash.o Python/pyhash.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pylifecycle.o Python/pylifecycle.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pymath.o Python/pymath.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pystate.o Python/pystate.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pythonrun.o Python/pythonrun.c
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/pytime.o Python/pytime.c
In file included from Python/pylifecycle.c:64:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/os/log.h:17:2: error: #error using os/log.h requires Xcode 8 or later
17 | #error using os/log.h requires Xcode 8 or later
| ^~~~~
/opt/local/bin/gcc-mp-14 -c -fno-strict-overflow -Wsign-compare -fno-common -dynamic -DNDEBUG -g -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -flto -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include/internal/mimalloc -I. -I./Include -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPy_BUILD_CORE -o Python/qsbr.o Python/qsbr.c
Python/pytime.c: In function 'py_get_system_clock':
Python/pytime.c:940:20: warning: unused variable 'tv' [-Wunused-variable]
940 | struct timeval tv;
| ^~
Python/pylifecycle.c: In function 'apple_log_write_impl':
Python/pylifecycle.c:2973: warning: ignoring '#pragma clang diagnostic' [-Wunknown-pragmas]
2973 | os_log_with_type(OS_LOG_DEFAULT, logtype, "%s", text);
Python/pylifecycle.c:2973: warning: ignoring '#pragma clang diagnostic' [-Wunknown-pragmas]
Python/pylifecycle.c:2973: warning: ignoring '#pragma clang diagnostic' [-Wunknown-pragmas]
Python/pylifecycle.c:2973:5: warning: 'internal_linkage' attribute directive ignored [-Wattributes]
2973 | os_log_with_type(OS_LOG_DEFAULT, logtype, "%s", text);
| ^~~~~~~~~~~~~~~~
In file included from /opt/local/lib/gcc14/gcc/x86_64-apple-darwin19/14.2.0/include-fixed/os/trace.h:17,
from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/os/log.h:14:
Python/pylifecycle.c:2973:5: error: implicit declaration of function '__builtin_os_log_format_buffer_size' [-Wimplicit-function-declaration]
2973 | os_log_with_type(OS_LOG_DEFAULT, logtype, "%s", text);
| ^~~~~~~~~~~~~~~~
Python/pylifecycle.c:2973:5: error: implicit declaration of function '__builtin_os_log_format'; did you mean '__builtin_isnormal'? [-Wimplicit-function-declaration]
2973 | os_log_with_type(OS_LOG_DEFAULT, logtype, "%s", text);
| ^~~~~~~~~~~~~~~~
Python/pylifecycle.c:2973: warning: ignoring '#pragma clang diagnostic' [-Wunknown-pragmas]
2973 | os_log_with_type(OS_LOG_DEFAULT, logtype, "%s", text);
make: *** [Python/pylifecycle.o] Error 1
Catalina has Xcode 12 though:
% /usr/bin/xcodebuild -version
Xcode 12.4
Build version 12D4e
Is this Apple bug?
CPython versions tested on:
3.13
Operating systems tested on:
macOS
While Xcode 12.4 may be installed on your build machine, the snippet you provide shows you are using a compiler not provided by Xcode: /opt/local/bin/gcc-mp-14 which is presumably from MacPorts gcc14. We generally only build and test on macOS with Apple-supplied compilers for macOS. But feel free to submit a PR for consideration if there is a straight-forward fix.
I think it's because gcc-mp-14 is picked up first due to it's location in the path (for instance, gcc is picked up by default over clang for me). So use ./configure CC=... where you can replace CC=... by the OS-provided compiler instead of a custom one.
I don't think there is an easy patch as we would not be able to test it continuously so I suggest to close this issue.
It should be not be hard to set up CI on GH in fact. And it is strange that one of the two major compilers is completely unsupported :(
As per PEP-11, we only support clang on Apple. The bug seems to occur because of some builtins that are not available on the GCC side so it might even be possible that we would have aborted the compilation earlier.
@picnixz I know maintainers of gcc for Darwin, if it is their bug, I will request them to address it. Let’s fix building Python with gcc, it is certainly possible.
I think it's an issue on their side but it can also be an issue on our side where the libraries are incorrectly picked up or if we're missing an include. Could you ask them where this function should be located actually?
The reason why I think it's missing is because of:
Python/pylifecycle.c:2973:5: error: implicit declaration of function '__builtin_os_log_format_buffer_size' [-Wimplicit-function-declaration]
Let’s fix building Python with gcc, it is certainly possible.
It could but we should be aware of all the other existing possible differences with clang. It's definitely not an easy task to maintain it and it requires a core developer to react to such issues. Officially supporting GCC on Apple would require more than just fixes I think.
I think it's an issue on their side but it can also be an issue on our side where the libraries are incorrectly picked up or if we're missing an include. Could you ask them where this function should be located actually?
The reason why I think it's missing is because of:
Python/pylifecycle.c:2973:5: error: implicit declaration of function '__builtin_os_log_format_buffer_size' [-Wimplicit-function-declaration]Let’s fix building Python with gcc, it is certainly possible.
It could but we should be aware of all the other existing possible differences with clang. It's definitely not an easy task to maintain it and it requires a core developer to react to such issues. Officially supporting GCC on Apple would require more than just fixes I think.
IMHO changes to CPython for GCC on macOS should not include workarounds for limitations in GCC that cause it to not be able to parse Apple's headers. The error above seems to be related to using an available system API in a documented way, but compilation fails due to system headers requiring features that GCC does not (yet) support.
Just supporting Apple's toolchain on the range of macOS versions we support gets hairy enough without considering workarounds for GCC.
A better way forward here would IMHO be to tweak configure.ac such that on Apple platforms the system compiler is preferred (either by defaulting to clang or to /usr/bin/xcrun clang (on recent enough versions, I don't recall in which version the xcrun command was introduced)
