botan
botan copied to clipboard
randombit
Latest master does not build on MacOS
I just updated my Botan repo to the latest version, but am now encountering a problem when building on MacOS:
INFO: configure.py invoked with options "--library-suffix=.macOS.Debug --no-optimizations --debug-mode --link-method=copy --disable-armv8crypto --disable-neon --cc-abi-flags=-force_cpusubtype_ALL -arch x86_64 -arch arm64 --with-external-includedir=/Users/james/Work/3rdParty/botan/Build/../../zlib/Source --no-install-python-module --disable-shared-library --without-documentation --build-targets=static --includedir=/Users/james/Work/3rdParty/botan/Build/../include/macos --prefix=/Users/james/Work/3rdParty/botan/Build/Output/macos/Debug"
INFO: Configuring to build Botan 3.9.0 (revision git:7964d1f52209a2f704471b1dc4462db65448cb23)
INFO: Python version: "3.13.3 (main, Apr 8 2025, 13:54:08) [Clang 16.0.0 (clang-1600.0.26.6)]"
INFO: Autodetected platform information: OS="Darwin" machine="arm64" proc="arm"
INFO: Guessing target OS is darwin (use --os to set)
INFO: Guessing to use compiler xcode (use --cc or CXX to set)
INFO: Guessing target processor is a arm64 (use --cpu to set)
INFO: Using /etc/ssl/cert.pem as system certificate store
INFO: Auto-detected compiler version xcode 16.0
INFO: Auto-detected compiler arch arm64
INFO: Target is xcode:16.0-macos-arm64
INFO: Skipping (dependency failure): asio certstor_sqlite3 chacha_simd32 noekeon_simd serpent_simd sessions_sqlite3 shacal2_simd tpm2_crypto_backend tpm2_ecc tpm2_rsa
INFO: Skipping (incompatible CPU): aes_armv8 aes_ni aes_power8 aes_vaes aes_vperm argon2_avx2 argon2_ssse3 camellia_gfni chacha_avx2 chacha_avx512 cpuid_arm32 cpuid_loongarch64 cpuid_ppc cpuid_riscv64 cpuid_x86 ghash_cpu ghash_vperm idea_sse2 keccak_perm_bmi2 processor_rng rdseed serpent_avx2 serpent_avx512 sha1_armv8 sha1_avx2 sha1_simd sha1_x86 sha2_32_armv8 sha2_32_avx2 sha2_32_simd sha2_32_x86 sha2_64_avx2 sha2_64_avx512 sha2_64_x86 shacal2_armv8 shacal2_avx2 shacal2_avx512 shacal2_x86 simd_2x64 simd_4x32 simd_4x64 simd_8x64 simd_avx2 simd_avx512 sm4_gfni sm4_x86 zfec_sse2 zfec_vperm
INFO: Skipping (incompatible OS): certstor_system_windows win32_stats
INFO: Skipping (incompatible compiler): sm4_armv8
INFO: Skipping (no enabled compression schemes): compression
INFO: Skipping (not requested): jitter_rng
INFO: Skipping (requires external dependency): boost bzip2 commoncrypto esdm_rng lzma sqlite3 tpm tpm2 zlib
WARNING: These modules are deprecated and will be removed in a future release (consider disabling with --disable-deprecated-features): adler32 crc32 cryptobox dilithium_aes dlies gost_28147 gost_3410 keccak kyber_90s legacy_ec_point lion mce md4 md5 noekeon prf_x942 shake_cipher siphash streebog x919_mac
INFO: Loading modules: adler32 aead aes aes_crystals_xof argon2 argon2fmt aria asn1 auto_rng base base32 base58 base64 bcrypt bcrypt_pbkdf bigint bitvector blake2 blake2mac blake2s blinding block blowfish camellia cascade cast128 cbc ccm certstor_flatfile certstor_sql certstor_system certstor_system_macos cfb chacha chacha20poly1305 chacha_rng checksum classic_mceliece cmac comb4p cpuid cpuid_aarch64 crc24 crc32 cryptobox cshake_xof ctr curve448 des dh dilithium dilithium_aes dilithium_common dilithium_round3 dilithium_shake dl_algo dl_group dlies dsa dyn_load eax ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 ed448 elgamal eme_oaep eme_pkcs1 eme_raw emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 frodokem frodokem_aes frodokem_common gcm getentropy ghash gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp hss_lms http_util hybrid_kem idea iso9796 kdf kdf1 kdf1_iso18033 kdf2 keccak keccak_perm kex_to_kem_adapter keypair kmac kuznyechik kyber kyber_90s kyber_common kyber_round3 legacy_ec_point lion locking_allocator mac mce md4 md5 mdx_hash mem_pool mgf1 ml_dsa ml_kem mode_pad modes mp nist_keywrap noekeon numbertheory ocb ofb os_utils par_hash passhash9 pbes2 pbkdf pbkdf2 pcurves pcurves_brainpool256r1 pcurves_brainpool384r1 pcurves_brainpool512r1 pcurves_frp256v1 pcurves_generic pcurves_impl pcurves_numsp512d1 pcurves_secp192r1 pcurves_secp224r1 pcurves_secp256k1 pcurves_secp256r1 pcurves_secp384r1 pcurves_secp521r1 pcurves_sm2p256v1 pem pgp_s2k pk_pad pkcs11 poly1305 poly_dbl pqcrystals prf_tls prf_x942 psk_db pubkey raw_hash rc4 rfc3394 rfc6979 rmd160 rng roughtime rsa salsa20 scrypt seed serpent sessions_sql sha1 sha2_32 sha2_64 sha2_64_armv8 sha3 shacal2 shake shake_cipher shake_xof siphash siv skein slh_dsa_sha2 slh_dsa_shake sm2 sm3 sm4 socket sodium sp800_108 sp800_56a sp800_56c sphincsplus_common sphincsplus_sha2 sphincsplus_sha2_base sphincsplus_shake sphincsplus_shake_base srp6 stateful_rng stream streebog system_rng thread_utils threefish_512 tls tls12 tls13 tls13_pqc tls_cbc tree_hash trunc_hash tss twofish utils uuid whirlpool x25519 x448 x509 x919_mac xmd xmss xof xts zfec
INFO: Using copy to link files into build dir (use --link-method to change)
INFO: Botan 3.9.0 (revision git:7964d1f52209a2f704471b1dc4462db65448cb23) (unreleased undated) build setup is complete
The first few dozen or so files build ok, but then I hit this:
clang++ -fPIC -fstack-protector -pthread -stdlib=libc++ -force_cpusubtype_ALL -arch x86_64 -arch arm64 -std=c++20 -D_REENTRANT -g -Wall -Wextra -Wpedantic -Wshadow -Wstrict-aliasing -Wstrict-overflow=5 -Wcast-align -Wmissing-declarations -Wpointer-arith -Wcast-qual -Wshorten-64-to-32 -Wcomma -Wdocumentation -DBOTAN_IS_BEING_BUILT -march=armv8.2-a+sha3 -I build/include/public -I build/include/internal -isystem build/include/external -isystem /Users/james/Work/3rdParty/zlib/Source -c src/lib/hash/sha2_64/sha2_64_armv8/sha2_64_armv8.cpp -o build/obj/lib/hash_sha2_64_sha2_64_armv8.o
error: unknown target CPU 'armv8.2-a+sha3'
note: valid target CPU values are: nocona, core2, penryn, bonnell, atom, silvermont, slm, goldmont, goldmont-plus, tremont, nehalem, corei7, westmere, sandybridge, corei7-avx, ivybridge, core-avx-i, haswell, core-avx2, broadwell, skylake, skylake-avx512, skx, cascadelake, cooperlake, cannonlake, icelake-client, rocketlake, icelake-server, tigerlake, sapphirerapids, alderlake, raptorlake, meteorlake, sierraforest, grandridge, graniterapids, graniterapids-d, emeraldrapids, knl, knm, k8, athlon64, athlon-fx, opteron, k8-sse3, athlon64-sse3, opteron-sse3, amdfam10, barcelona, btver1, btver2, bdver1, bdver2, bdver3, bdver4, znver1, znver2, znver3, znver4, x86-64, x86-64-v2, x86-64-v3, x86-64-v4
make: *** [build/obj/lib/hash_sha2_64_sha2_64_armv8.o] Error 1
make: *** Waiting for unfinished jobs....
This is kind of weird as based on the list of supported CPUs this is an x86 only compiler
That said the option --disable-armv8crypto should in any case be preventing trying to use these instructions fa28fc6 should resolve.
Unfortunately that didn't seem to solve anything:
src/lib/utils/cpuid/cpuid_aarch64/cpuid_aarch64.cpp:120:11: error: unknown use of instruction mnemonic without a size suffix
120 | asm("and v0.16b, v0.16b, v0.16b");
| ^
<inline asm>:1:2: note: instantiated into assembly here
1 | and v0.16b, v0.16b, v0.16b
| ^
src/lib/utils/cpuid/cpuid_aarch64/cpuid_aarch64.cpp:124:11: error: out of range literal value
124 | asm(".word 0x4e284800");
| ^
<inline asm>:1:8: note: instantiated into assembly here
1 | .word 0x4e284800
| ^
src/lib/utils/cpuid/cpuid_aarch64/cpuid_aarch64.cpp:128:11: error: out of range literal value
128 | asm(".word 0x0ee0e000");
| ^
<inline asm>:1:8: note: instantiated into assembly here
1 | .word 0x0ee0e000
| ^
src/lib/utils/cpuid/cpuid_aarch64/cpuid_aarch64.cpp:132:11: error: out of range literal value
132 | asm(".word 0x5e280800");
| ^
<inline asm>:1:8: note: instantiated into assembly here
1 | .word 0x5e280800
| ^
src/lib/utils/cpuid/cpuid_aarch64/cpuid_aarch64.cpp:136:11: error: out of range literal value
136 | asm(".word 0x5e282800");
| ^
<inline asm>:1:8: note: instantiated into assembly here
1 | .word 0x5e282800
| ^
5 errors generated.
make: *** [build/obj/lib/utils_cpuid_aarch64.o] Error 1
Sorry this got missed. @randombit please don't hesitate tagging (or assigning) me on macOS related tickets.
Since recently, --disable-... got removed, please try this: --disable-modules="$(./configure.py --list-modules | grep armv8 | paste -sd, -)" which should disable all armv8-specific modules. For me that compiles and runs fine (on an M2 MacBook with Sequoia).
Once 4966 lands, the above one-liner should become easier, like: --disable-modules='*armv8'.
Great. Hopefully I'll be back on to Mac stuff soon and I can try it out.
To be honest though, the only reason I disabled ARM intrinsics in the first place was because I couldn't get Botan to build universal binaries on my ARM based Mac. (It's entirely possible I was doing something wrong in the first place, but I honestly can't remember.)
Strange... My config didn't disable any feature or module - and I've been building Botan master successfully on MacOS Sequoia 15.5 (Xcode-16.4) on both Intel-based and Apple Silicon M2 Max.
