janestreet
Build failure with clang: unsupported `-mpopcnt` option
#=== ERROR while compiling base.v0.16.3 =======================================#
# context 2.2.0~beta3~dev | linux/arm64 | ocaml.5.1.1 | https://opam.ocaml.org#2b6e600e
# path ~/.opam/default-clang/.opam-switch/build/base.v0.16.3
# command ~/.opam/opam-init/hooks/sandbox.sh build dune build -p base -j 7
# exit-code 1
# env-file ~/.opam/log/base-12454-0cee78.env
# output-file ~/.opam/log/base-12454-0cee78.out
### output ###
# [...]
# (cd _build/default/src && /usr/bin/clang -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_FILE_OFFSET_BITS=64 -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_LARGEFILE64_SOURCE -mpopcnt -g -I /home/antonin/.opam/default-clang/lib/ocaml -I /home/antonin/.opam/default-clang/lib/sexplib0 -I ../compiler-stdlib/src -I ../hash_types/src -I ../shadow-stdlib/src -o exn_stubs.o -c exn_stubs.c)
# clang: error: unsupported option '-mpopcnt' for target 'aarch64-redhat-linux-gnu'
# File "src/dune", line 14, characters 10-21:
# 14 | (c_names bytes_stubs exn_stubs int_math_stubs hash_stubs am_testing)
# ^^^^^^^^^^^
# (cd _build/default/src && /usr/bin/clang -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_FILE_OFFSET_BITS=64 -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_LARGEFILE64_SOURCE -mpopcnt -g -I /home/antonin/.opam/default-clang/lib/ocaml -I /home/antonin/.opam/default-clang/lib/sexplib0 -I ../compiler-stdlib/src -I ../hash_types/src -I ../shadow-stdlib/src -o bytes_stubs.o -c bytes_stubs.c)
# clang: error: unsupported option '-mpopcnt' for target 'aarch64-redhat-linux-gnu'
# File "src/dune", line 14, characters 47-57:
# 14 | (c_names bytes_stubs exn_stubs int_math_stubs hash_stubs am_testing)
# ^^^^^^^^^^
# (cd _build/default/src && /usr/bin/clang -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_FILE_OFFSET_BITS=64 -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -D_LARGEFILE64_SOURCE -mpopcnt -g -I /home/antonin/.opam/default-clang/lib/ocaml -I /home/antonin/.opam/default-clang/lib/sexplib0 -I ../compiler-stdlib/src -I ../hash_types/src -I ../shadow-stdlib/src -o hash_stubs.o -c hash_stubs.c)
# clang: error: unsupported option '-mpopcnt' for target 'aarch64-redhat-linux-gnu'
It seems that clang doesn't need the flag and that __builtin_popcountg should be used.
Getting the same error on Termux (where gcc is linked to clang):
#=== ERROR while compiling base.v0.17.0 =======================================#
# context 2.1.6 | linux/arm64 | ocaml-base-compiler.5.2.0 | https://opam.ocaml.org#6c0498c5
# path ~/.opam/5.2.0/.opam-switch/build/base.v0.17.0
# command ~/.opam/5.2.0/bin/dune build -p base -j 7
# exit-code 1
# env-file ~/.opam/log/base-22295-f950c4.env
# output-file ~/.opam/log/base-22295-f950c4.out
### output ###
# [...]
# (cd _build/default/src && /data/data/com.termux/files/usr/bin/gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -pthread -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -mpopcnt -g -I /data/data/com.termux/files/home/.opam/5.2.0/lib/ocaml -I /data/data/com.termux/files/home/.opam/5.2.0/lib/ocaml_intrinsics_kernel -I /data/data/com.termux/files/home/.opam/5.2.0/lib/sexplib0 -I ../hash_types/src[...]
# gcc: error: unsupported option '-mpopcnt' for target 'aarch64-unknown-linux-android24'
# File "src/dune", line 19, characters 57-66:
# 19 | (names bytes_stubs exn_stubs int_math_stubs hash_stubs obj_stubs am_testing)
# ^^^^^^^^^
# (cd _build/default/src && /data/data/com.termux/files/usr/bin/gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -pthread -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -mpopcnt -g -I /data/data/com.termux/files/home/.opam/5.2.0/lib/ocaml -I /data/data/com.termux/files/home/.opam/5.2.0/lib/ocaml_intrinsics_kernel -I /data/data/com.termux/files/home/.opam/5.2.0/lib/sexplib0 -I ../hash_types/src[...]
# gcc: error: unsupported option '-mpopcnt' for target 'aarch64-unknown-linux-android24'
Also termux (Android 11) user here
I have worked around this by cloning and editing the sources
git clone https://github.com/janestreet/base.git
cd base
git checkout v0.17.1
Then edit <src/discover/discover.ml> to remove the "-mpopcnt" strings.
Now you can
opam pin edit base
# Change src to point to your checkout
Compiled fine and then I can use Base.Int.popcount (or indeed install ocamlformat)
I'm also running into this now after upgrading to Clang 16 + building on a 5.3.0+tsan switch.
Then edit <src/discover/discover.ml> to remove the
"-mpopcnt"strings.
And how does the edited discover.ml look like?
@anta40
I just did the simplest da" vim motion, the file could definitely be further simplified.
diff --git a/src/discover/discover.ml b/src/discover/discover.ml
index 47786c0..e2e6052 100644
--- a/src/discover/discover.ml
+++ b/src/discover/discover.ml
@@ -15,6 +15,6 @@ let () =
~name:"discover"
~args:[ "-o", Set_string output, "FILENAME output file" ]
(fun c ->
- let has_popcnt = c_test c ~c_flags:[ "-mpopcnt" ] program in
- Flags.write_sexp !output (if has_popcnt then [ "-mpopcnt" ] else []))
+ let has_popcnt = c_test c ~c_flags:[] program in
+ Flags.write_sexp !output (if has_popcnt then [] else []))
;;
But what it needs as a proper fix is figuring out why the build system thinks it has popcount when it doesn't.
But now I can't reproduce the failure on my Android – clang did get updated since though so maybe that's why
I just did the simplest
da"vim motion, the file could definitely be further simplified.
I copy pasted your update and... couldn't build it:
[ERROR] The compilation of base.v0.17.1 failed at "dune build -p base -j 7".
#=== ERROR while compiling base.v0.17.1 =======================================#
# context 2.2.1 | macos/arm64 | ocaml-option-flambda.1 ocaml-variants.5.3.0+options | pinned(file:///Users/andretampubolon/Codes/OCaml/base)
# path ~/.opam/5.3.0-flambda/.opam-switch/build/base.v0.17.1
# command ~/.opam/opam-init/hooks/sandbox.sh build dune build -p base -j 7
# exit-code 1
# env-file ~/.opam/log/base-9804-d10ae5.env
# output-file ~/.opam/log/base-9804-d10ae5.out
### output ###
# [...]
# (cd _build/default && /Users/andretampubolon/.opam/5.3.0-flambda/bin/ocamlopt.opt -w -40 -g -a -o hash_types/src/base_internalhash_types.cmxa -cclib -lbase_internalhash_types_stubs hash_types/src/.base_internalhash_types.objs/native/base_internalhash_types.cmx)
# ar: error: couldn't create cache file '/var/folders/42/7rwmfsfs4rl6jv1xl12n88gc0000gp/T/xcrun_db-7jgPaFov' (errno=Operation not permitted)
# ar: error: couldn't create cache file '/var/folders/42/7rwmfsfs4rl6jv1xl12n88gc0000gp/T/xcrun_db-flNkzlfr' (errno=Operation not permitted)
# (cd _build/default && /Users/andretampubolon/.opam/5.3.0-flambda/bin/ocamlmklib -g -o hash_types/src/base_internalhash_types_stubs hash_types/src/internalhash_stubs.o)
# ar: error: couldn't create cache file '/var/folders/42/7rwmfsfs4rl6jv1xl12n88gc0000gp/T/xcrun_db-LvhdveQR' (errno=Operation not permitted)
# ar: error: couldn't create cache file '/var/folders/42/7rwmfsfs4rl6jv1xl12n88gc0000gp/T/xcrun_db-DEaSRi2u' (errno=Operation not permitted)
# (cd _build/default && /Users/andretampubolon/.opam/5.3.0-flambda/bin/ocamlopt.opt -w -40 -g -a -o shadow-stdlib/src/shadow_stdlib.cmxa shadow-stdlib/src/.shadow_stdlib.objs/native/shadow_stdlib.cmx)
# ar: error: couldn't create cache file '/var/folders/42/7rwmfsfs4rl6jv1xl12n88gc0000gp/T/xcrun_db-hQuhmNBe' (errno=Operation not permitted)
# ar: error: couldn't create cache file '/var/folders/42/7rwmfsfs4rl6jv1xl12n88gc0000gp/T/xcrun_db-JggNswB5' (errno=Operation not permitted)
# (cd _build/default && /Users/andretampubolon/.opam/5.3.0-flambda/bin/ocamlopt.opt -w -40 -g -a -o src/base.cmxa -cclib -lbase_stubs src/.base.objs/native/base__.cmx src/.base.objs/native/base__Bool0.cmx src/.base.objs/native/base__Globalize.cmx src/.base.objs/native/base__Poly0.cmx src/.base.objs/native/base__Import0.cmx src/.base.objs/native/base__Int0.cmx src/.base.objs/native/base__Printf.[...]
# ar: error: couldn't create cache file '/var/folders/42/7rwmfsfs4rl6jv1xl12n88gc0000gp/T/xcrun_db-PH0KIMQA' (errno=Operation not permitted)
# ar: error: couldn't create cache file '/var/folders/42/7rwmfsfs4rl6jv1xl12n88gc0000gp/T/xcrun_db-2bNejEaB' (errno=Operation not permitted)
There are some Operation not permitted lines there. My macOS account is non-admin. Wonder if that's the cause hmm...
Yes this was specifically about the -mpopcnt flag, it seems like your error messages don't have anything about popcount so you have a different issue.
It's probably best to open a different issue to avoid confusion?
Fixed in https://github.com/ocaml/opam-repository/pull/27716 and https://github.com/janestreet/base/pull/180
I got this from homebrew package build,
#=== ERROR while compiling base.v0.17.1 =======================================#
# context 2.3.0 | macos/arm64 | ocaml-variants.5.2.0+options | https://opam.ocaml.org#8106bcf57fc33[218](https://github.com/Homebrew/homebrew-core/actions/runs/14719648582/job/41311047166?pr=221799#step:3:219)275bbf0d02e725f4e9fcd0f6
# path /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/.opam-switch/build/base.v0.17.1
# command /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/bin/dune build -p base -j 3
# exit-code 1
# env-file /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T/tmp.IFAlmoHupS/log/base-5070-5ec6d2.env
# output-file /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T/tmp.IFAlmoHupS/log/base-5070-5ec6d2.out
### output ###
# [...]
# (cd _build/default/src && /opt/homebrew/Library/Homebrew/shims/mac/super/cc -O2 -fno-strict-aliasing -fwrapv -pthread -pthread -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -D_LARGEFILE64_SOURCE -mpopcnt -g -I /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/lib/ocaml -I /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/lib/ocaml_intrinsics_kernel -I /private/tmp/flow-202[...]
# clang: error: unsupported option '-mpopcnt' for target 'arm64-apple-darwin24.4.0'
# File "src/dune", line 19, characters 57-66:
# 19 | (names bytes_stubs exn_stubs int_math_stubs hash_stubs obj_stubs am_testing)
# ^^^^^^^^^
# (cd _build/default/src && /opt/homebrew/Library/Homebrew/shims/mac/super/cc -O2 -fno-strict-aliasing -fwrapv -pthread -pthread -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -D_LARGEFILE64_SOURCE -mpopcnt -g -I /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/lib/ocaml -I /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/lib/ocaml_intrinsics_kernel -I /private/tmp/flow-202[...]
# clang: error: unsupported option '-mpopcnt' for target 'arm64-apple-darwin24.4.0'
# (cd _build/default && /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/bin/ocamlopt.opt -w -40 -g -I src/.base.objs/byte -I src/.base.objs/native -I /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/lib/ocaml_intrinsics_kernel -I /private/tmp/flow-20250428-4929-89lp9d/flow-0.269.1/_opam/lib/sexplib0 -I hash_types/src/.base_internalhash_types.objs/byte -I hash_types/src/.base_[...]
# File "src/array.ml", line 399, characters 28-75:
# 399 | result := Bool.select ((keep_left_if [@inlined]) (compare x !result)) x !result
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Warning 55 [inlining-impossible]: Cannot inline: Unknown function
should be reopen?
Apple clang 17
Looks like OCaml base 17.1 doesn't contain https://github.com/janestreet/base/pull/180 -- so no it doesn't indicate the fix isn't working, just that the fix hasn't made its way into a release. You could pin it to the latest of the v0.17 branch which just contains that fix https://github.com/janestreet/base/commits/v0.17/
See also https://github.com/janestreet/base/issues/164#issuecomment-2629138891
It seems that this is still happening on macos 15. I have tried with base v0.17.2 now.
#=== ERROR while compiling base.v0.17.2 =======================================#
# context 2.3.0 | macos/arm64 | ocaml-variants.5.2.0+options | [https://opam.ocaml.org#e28c86445c807c3d11aecedb95a03661481a7d1c](https://opam.ocaml.org/#e28c86445c807c3d11aecedb95a03661481a7d1c)
# path /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/.opam-switch/build/base.v0.17.2
# command /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/bin/dune build -p base -j 3
# exit-code 1
# env-file /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T/tmp.BaMWNerlpS/log/base-5035-73c973.env
# output-file /private/var/folders/wp/mgkns75d6dd_kq5800l78p080000gq/T/tmp.BaMWNerlpS/log/base-5035-73c973.out
### output ###
# [...]
# (cd _build/default/src && /opt/homebrew/Library/Homebrew/shims/mac/super/cc -O2 -fno-strict-aliasing -fwrapv -pthread -pthread -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -D_LARGEFILE64_SOURCE -mpopcnt -g -I /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/lib/ocaml -I /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/lib/ocaml_intrinsics_kernel -I /private/tmp/flow-202[...]
# clang: error: unsupported option '-mpopcnt' for target 'arm64-apple-darwin24.4.0'
# File "src/dune", line 19, characters 57-66:
# 19 | (names bytes_stubs exn_stubs int_math_stubs hash_stubs obj_stubs am_testing)
# ^^^^^^^^^
# (cd _build/default/src && /opt/homebrew/Library/Homebrew/shims/mac/super/cc -O2 -fno-strict-aliasing -fwrapv -pthread -pthread -D_FILE_OFFSET_BITS=64 -fdiagnostics-color=always -D_LARGEFILE64_SOURCE -mpopcnt -g -I /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/lib/ocaml -I /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/lib/ocaml_intrinsics_kernel -I /private/tmp/flow-202[...]
# clang: error: unsupported option '-mpopcnt' for target 'arm64-apple-darwin24.4.0'
# (cd _build/default && /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/bin/ocamlopt.opt -w -40 -g -I src/.base.objs/byte -I src/.base.objs/native -I /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/lib/ocaml_intrinsics_kernel -I /private/tmp/flow-20250506-4924-yyclwd/flow-0.269.1/_opam/lib/sexplib0 -I hash_types/src/.base_internalhash_types.objs/byte -I hash_types/src/.base_[...]
# File "src/array.ml", line 399, characters 28-75:
# 399 | result := Bool.select ((keep_left_if [@inlined]) (compare x !result)) x !result
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Warning 55 [inlining-impossible]: Cannot inline: Unknown function
We are encountering this on Semgrep. It's blocking the release of Semgrep on Homebrew. I initially assumed that upgrading to Base 0.17.2 would solve the problem but it did not. You should be able to repro with brew install --head semgrep on macOS 15.4.1.
I haven't been able to reproduce this on my own Mac yet, I'll keep investigating.
I ran into this issue today using on macOS 15.4.1 + ocaml 5.2.1. Upgrading to 5.3.0 (with base 0.17.2) on a new opam switch did fix it. I cleared the opam cache before too, just in case.
@nmote Although, I can confirm it still failed during semgrep installation, even though it looks like its also 5.3.0 + 17.2.
It's probably also worth noting that I have no problem installing Base locally if I just create a new opam switch myself and install it there (OCaml 5.3.0 plus Base 0.17.2). But brew install --head semgrep still leads to the error, despite the versions being ostensibly the same. I am not sure what is different.
@dkalinichenko-js I've mostly figured out what is going on here. The initial fix worked around the Clang bug that is present in Apple Clang 17.0.0, the current version of Clang on macOS, by manually invoking cc with the -mpopcnt flag, ensuring that no other flags were present so as not to trigger the bug. This usually works.
Unfortunately, during installation, Homebrew puts a different cc on the PATH, which is actually a Ruby script that wraps invocations to cc. In particular (at least on my machine) it adds -W1,-headerpad_max_install_names, which is enough for the Clang bug to manifest. Compilation of the small test program succeeds incorrectly even with the -mpopcnt flag, leading Base to believe that -mpopcnt should be enabled.
For some reason, this Clang bug does not manifest during build time (despite the additional flags that Homebrew tacks on), so Clang (correctly, but annoyingly) errors out, informing us that -mpopcnt is invalid for ARM targets.
This is affecting the release of Semgrep on Homebrew (e.g. 1.124.0) and is also preventing Flow from releasing (e.g. 0.272.2). It is likely affecting any other OCaml projects that release to Homebrew.
I am not sure how best to address this, but one suggestion I have is to use Dune Configurator to check the architecture and disable the -mpopcnt flag for arm64 rather than testing the compiler. It's certainly less palatable than using a feature test, but given that we have to rely on a buggy version of Clang, it would be more reliable. It does not look like Clang needs any special flag to generate the CNT instruction on ARM, so such a gate should be harmless.
Thanks! Do you have a specific timeline in mind? We are weighing whether to wait for a Base release with a fix or putting together some unpleasant workaround.