`std::signbit` cannot be found when building on Apple M1

[Yuhta]

When building (make debug) on Apple M1, I run into these errors:

/Library/Developer/CommandLineTools/usr/bin/make cmake BUILD_DIR=debug BUILD_TYPE=Debug
mkdir -p _build/debug && \
	cmake -B \
		"_build/debug" \
		-DTREAT_WARNINGS_AS_ERRORS=1 -DENABLE_ALL_WARNINGS=1 -DVELOX_BUILD_MINIMAL="OFF" -DVELOX_BUILD_TESTING="ON" -DCMAKE_BUILD_TYPE=Debug \
		-GNinja -DMAX_LINK_JOBS= -DMAX_HIGH_MEM_JOBS= \
		-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
		 \
		
-- Build type: Debug
-- Disabling Codegen
-- Found Bison keg installed by Homebrew at /opt/homebrew/opt/bison
-- Found Flex keg installed by Homebrew at /opt/homebrew/opt/flex
-- xsimd v9.0.1
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/jimmylu/src/velox/_build/debug
/Library/Developer/CommandLineTools/usr/bin/make build BUILD_DIR=debug -j 10
cmake --build _build/debug -j 10
[1/909] Building CXX object third_party/googletest/googlemock/CMakeFiles/gmock_main.dir/src/gmock_main.cc.o
[2/909] Building CXX object third_party/googletest/googlemock/CMakeFiles/gmock.dir/src/gmock-all.cc.o
[3/909] Building CXX object third_party/googletest/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[4/909] Linking CXX static library lib/libgtestd.a
[5/909] Linking CXX static library lib/libgmockd.a
[6/909] Linking CXX static library lib/libgmock_maind.a
[7/909] Linking CXX static library lib/libgtest_maind.a
[8/909] Building CXX object velox/common/base/CMakeFiles/velox_common_base.dir/BitUtil.cpp.o
FAILED: velox/common/base/CMakeFiles/velox_common_base.dir/BitUtil.cpp.o 
ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DGFLAGS_IS_A_DLL=0 -I/opt/homebrew/opt/icu4c/include -I/Users/jimmylu/src/velox/. -I/Users/jimmylu/src/velox/velox/external/xxhash -I/Users/jimmylu/src/velox/third_party/xsimd/include -isystem /opt/homebrew/include -isystem /Users/jimmylu/src/velox/velox -isystem /Users/jimmylu/src/velox/velox/external -isystem /Users/jimmylu/src/velox/velox/external/duckdb -isystem /Users/jimmylu/src/velox/velox/external/duckdb/tpch/dbgen/include -isystem /usr/local/include -isystem /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include -isystem /opt/homebrew/opt/[email protected]/include -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -fvisibility=hidden -fvisibility-inlines-hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Werror -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis -Wno-mismatched-tags -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.0.sdk -mmacosx-version-min=12.6 -fPIC -std=gnu++17 -MD -MT velox/common/base/CMakeFiles/velox_common_base.dir/BitUtil.cpp.o -MF velox/common/base/CMakeFiles/velox_common_base.dir/BitUtil.cpp.o.d -o velox/common/base/CMakeFiles/velox_common_base.dir/BitUtil.cpp.o -c /Users/jimmylu/src/velox/velox/common/base/BitUtil.cpp
In file included from /Users/jimmylu/src/velox/velox/common/base/BitUtil.cpp:18:
In file included from /Users/jimmylu/src/velox/./velox/common/base/Exceptions.h:22:
/usr/local/include/fmt/format.h:1772:12: error: expected unqualified-id
  if (std::signbit(value)) {  // value < 0 is false for NaN so use signbit.
           ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/math.h:170:5: note: expanded from macro 'signbit'
    ( sizeof(x) == sizeof(float)  ? __inline_signbitf((float)(x))        \
    ^

% clang --version
Apple clang version 14.0.0 (clang-1400.0.29.201)
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Looks like a configuration issue with MacOSX SDK, is there any expert on this can provide some clue how we can solve this?

[mbasmanova]

CC: @kgpai

[mbasmanova]

CC: @raulcd @assignUser

[raulcd]

hi @Yuhta , thanks for the report. I don't have an M1, I will try to get one to test (or spin and AWS instance) in the meantime, do you know if this only happens for DEBUG or this is happening on make release too?

[assignUser]

This seems to be an issue when xcode is not installed, and a macro from math.h clashes with std::signbit I found several threads talking about this and installing xcode seems to solve it?

• This lays out a solution: https://github.com/nlohmann/json/issues/2087#issuecomment-624331050
• https://root-forum.cern.ch/t/compiling-6-12-4-error/27423/12
• (it was also a bug in ancient gcc)

[Yuhta]

@assignUser Thanks for the help, I do have -isysroot when the compiler is invoked:

ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DGFLAGS_IS_A_DLL=0 -I/opt/homebrew/opt/icu4c/include -I/Users/jimmylu/src/velox/. -I/Users/jimmylu/src/velox/velox/external/xxhash -I/Users/jimmylu/src/velox/third_party/xsimd/include -isystem /opt/homebrew/include -isystem /Users/jimmylu/src/velox/velox -isystem /Users/jimmylu/src/velox/velox/external -isystem /Users/jimmylu/src/velox/velox/external/duckdb -isystem /Users/jimmylu/src/velox/velox/external/duckdb/tpch/dbgen/include -isystem /usr/local/include -isystem /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include -isystem /opt/homebrew/opt/[email protected]/include -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -fvisibility=hidden -fvisibility-inlines-hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Werror -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis -Wno-mismatched-tags -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.0.sdk -mmacosx-version-min=12.6 -fPIC -std=gnu++17 -MD -MT velox/common/base/CMakeFiles/velox_common_base.dir/BitUtil.cpp.o -MF velox/common/base/CMakeFiles/velox_common_base.dir/BitUtil.cpp.o.d -o velox/common/base/CMakeFiles/velox_common_base.dir/BitUtil.cpp.o -c /Users/jimmylu/src/velox/velox/common/base/BitUtil.cpp

[Yuhta]

@raulcd It's also happening in release build

[CodingCat]

any solution to this?

[mbasmanova]

CC: @kgpai @raulcd

[assignUser]

@CodingCat @Yuhta This is not exactly an issue related to velox but rather with your build environment, which makes it hard to reproduce (especially as I don't have a mac :D ).

This SO post has several suggested fixes from a lot of users covering a lot of different configurations.

The main ones are setting -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.0.sdk/usr/include (note the usr/include is missing in your invocation :warning: ) removing and reinstalling xcode, removing the sdk folder...

[CodingCat]

you are right, I deleted all SDK and developer tools locally, and reinstalled....it get passed now

[leoluan2009]

you are right, I deleted all SDK and developer tools locally, and reinstalled....it get passed now

Hi @CodingCat , how do you resinstalled? After I executed below commands:

sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install

The same error still happened.

[darrenfu]

The main ones are setting -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.0.sdk/usr/include (note the usr/include is missing in your invocation ⚠️ ) removing and reinstalling xcode, removing the sdk folder...

I encountered such issue before. It's because the CLT's SDK version (e.g. 12.3) is incompatible with XCode's SDK (e.g. 13.0). My workaround is to force Clang to use XCode's SDK instead of Command Line Tool (CLT)'s SDK.

Here are the detail steps:

• Create a soft link to cheat clang to use XCode's SDK instead of CLT's sdk. Then CMake will always load the correct SDK version from XCode's SDK directory.

cd /Library/Developer/CommandLineTools/SDKs/
# Let's say the CLT directory path to load is MacOSX12.3.sdk/, we backup the origin CLT's SDK directory
sudo mv MacOSX12.3.sdk MacOSX12.3.sdk.bak
sudo ln -s $(xcrun --show-sdk-path) MacOSX12.3.sdk

I hope Apple can provide an ultimate solution to fix the CLT and XCode SDK incompatibility issue.

[leoluan2009]

@darrenfu thanks for your reply, you are right. But still have some another different errors.

Below is my detailed steps:

• remove old Command Line Tools by sudo rm -rf /Library/Developer/CommandLineTools
• download Installation package from https://download.developer.apple.com/Developer_Tools/Command_Line_Tools_for_Xcode_13.4/Command_Line_Tools_for_Xcode_13.4.dmg and install it

After install Command_Line_Tools_for_Xcode_13.4.dmg, the content in path /Library/Developer/CommandLineTools/SDKs/ is below:

ls -rlt /Library/Developer/CommandLineTools/SDKs/
total 0
drwxr-xr-x  7 root  wheel  224 Mar 31  2022 MacOSX12.3.sdk
drwxr-xr-x  7 root  wheel  224 Mar 31  2022 MacOSX11.3.sdk
lrwxr-xr-x  1 root  wheel   14 Jan 12 11:54 MacOSX12.sdk -> MacOSX12.3.sdk
lrwxr-xr-x  1 root  wheel   14 Jan 12 11:54 MacOSX11.sdk -> MacOSX11.3.sdk
lrwxr-xr-x  1 root  wheel   14 Jan 12 11:55 MacOSX.sdk -> MacOSX12.3.sdk

Then the above errors gone when building, but another different errors happened as below:

FAILED: velox/dwio/common/tests/CMakeFiles/velox_dwio_common_bitpack_decoder_benchmark.dir/BitPackDecoderBenchmark.cpp.o
ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DGFLAGS_IS_A_DLL=0 -DVELOX_ENABLE_PARQUET -DVELOX_ENABLE_S3 -I/Users/xuedongluan/github/leoluan2009/velox/. -I/Users/xuedongluan/github/leoluan2009/velox/velox/external/xxhash -I/Users/xuedongluan/github/leoluan2009/velox/third_party/xsimd/include -isystem /opt/homebrew/include -isystem /Users/xuedongluan/github/leoluan2009/velox/velox -isystem /Users/xuedongluan/github/leoluan2009/velox/velox/external -isystem /Users/xuedongluan/github/leoluan2009/velox/velox/external/duckdb -isystem /Users/xuedongluan/github/leoluan2009/velox/velox/external/duckdb/tpch/dbgen/include -isystem /usr/local/include -isystem /opt/homebrew/opt/[email protected]/include -isystem /Users/xuedongluan/github/leoluan2009/velox/third_party/googletest/googletest/include -isystem /Users/xuedongluan/github/leoluan2009/velox/third_party/googletest/googletest -isystem /Users/xuedongluan/github/leoluan2009/velox/_build/debug/third_party/arrow_ep/install/include -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Werror -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk -fPIE -std=gnu++17 -MD -MT velox/dwio/common/tests/CMakeFiles/velox_dwio_common_bitpack_decoder_benchmark.dir/BitPackDecoderBenchmark.cpp.o -MF velox/dwio/common/tests/CMakeFiles/velox_dwio_common_bitpack_decoder_benchmark.dir/BitPackDecoderBenchmark.cpp.o.d -o velox/dwio/common/tests/CMakeFiles/velox_dwio_common_bitpack_decoder_benchmark.dir/BitPackDecoderBenchmark.cpp.o -c /Users/xuedongluan/github/leoluan2009/velox/velox/dwio/common/tests/BitPackDecoderBenchmark.cpp
In file included from /Users/xuedongluan/github/leoluan2009/velox/velox/dwio/common/tests/BitPackDecoderBenchmark.cpp:20:
In file included from /Users/xuedongluan/github/leoluan2009/velox/./velox/dwio/common/tests/Lemire/bmipacking32.h:33:
In file included from /Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/immintrin.h:13:
In file included from /Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/x86gprintrin.h:15:
/Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/hresetintrin.h:42:27: error: invalid input constraint 'a' in asm
  __asm__ ("hreset $0" :: "a"(__eax));
                          ^
In file included from /Users/xuedongluan/github/leoluan2009/velox/velox/dwio/common/tests/BitPackDecoderBenchmark.cpp:20:
In file included from /Users/xuedongluan/github/leoluan2009/velox/./velox/dwio/common/tests/Lemire/bmipacking32.h:33:
In file included from /Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/immintrin.h:17:
/Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/mmintrin.h:33:5: error: use of undeclared identifier '__builtin_ia32_emms'; did you mean '__builtin_isless'?
    __builtin_ia32_emms();
    ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/v1/math.h:670:12: note: '__builtin_isless' declared here
    return isless(__lcpp_x, __lcpp_y);
           ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/math.h:545:22: note: expanded from macro 'isless'
#define isless(x, y) __builtin_isless((x),(y))
                     ^
In file included from /Users/xuedongluan/github/leoluan2009/velox/velox/dwio/common/tests/BitPackDecoderBenchmark.cpp:20:
In file included from /Users/xuedongluan/github/leoluan2009/velox/./velox/dwio/common/tests/Lemire/bmipacking32.h:33:
In file included from /Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/immintrin.h:17:
/Library/Developer/CommandLineTools/usr/lib/clang/13.1.6/include/mmintrin.h:33:25: error: too few arguments to function call, expected 2, have 0
    __builtin_ia32_emms();

[darrenfu]

@leoluan2009 To tell from the error message, an arm64 build shouldn't be including immintrin.h. Can you run clang --version to check your clang version?

[leoluan2009]

@leoluan2009 To tell from the error message, an arm64 build shouldn't be including immintrin.h. Can you run clang --version to check your clang version?

Hi @darrenfu my clang version is:

clang --version
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: arm64-apple-darwin21.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[assignUser]

@darrenfu it seems like immintrin.h is included unguarded here:

https://github.com/facebookincubator/velox/blob/main/velox/dwio/common/tests/Lemire/bmipacking32.h#L33

But that was last changed two month's ago and I doubt that this went undetected for so long so :shrug:

[652053395]

I met the same problem, did you solve it ? @leoluan2009