emscripten-core
Host CC cannot compile on Mac OS Monterey (12.2.1)
With a fresh install of emsdk latest (I also tried tot), the bundled host compiler(clang, not emcc) cannot compile hello world.
I believe the the System framework has moved in Monterey somewhere under /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/, and this is not in the default search paths of the provided clang.
The error it gives is ld: library not found for -lSystem
This prevents me from building various third party projects using emconfigure (in this case GMP).
Version of emscripten/emsdk:
chris@ip-172-17-20-167 emsdk % emcc -v
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.6-git (646de914928022fa83dff561c5b765e6e378c9a1)
clang version 15.0.0 (https://github.com/llvm/llvm-project 5acd9c49a85fb07d8cc04803c27e7aa1fb8c0211)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /Users/chris/dev/emsdk/upstream/bin
The following is a terminal session reproducing the issue:
chris@localhost dev % git clone https://github.com/emscripten-core/emsdk.git
<snip>
chris@localhost dev % cd emsdk
chris@localhost emsdk % ./emsdk install tot
<snip>
chris@localhost emsdk % ./emsdk activate tot
<snip>
chris@localhost emsdk % source emsdk_env.sh
<snip>
chris@localhost emsdk % emconfigure env | grep HOST_CC
configure: env
HOST_CC=/Users/chris/dev/emsdk/upstream/bin/clang
chris@localhost emsdk % cat - > main.c
int main() { return 0; }
chris@localhost emsdk % /Users/chris/dev/emsdk/upstream/bin/clang main.c
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
chris@localhost emsdk % /Users/chris/dev/emsdk/upstream/bin/clang -v main.c
clang version 15.0.0 (https://github.com/llvm/llvm-project 5acd9c49a85fb07d8cc04803c27e7aa1fb8c0211)
Target: x86_64-apple-darwin21.3.0
Thread model: posix
InstalledDir: /Users/chris/dev/emsdk/upstream/bin
"/Users/chris/dev/emsdk/upstream/bin/clang-15" -cc1 -triple x86_64-apple-macosx12.0.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -main-file-name main.c -mrelocation-model pic -pic-level 2 -mframe-pointer=all -ffp-contract=on -fno-rounding-math -funwind-tables=2 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu penryn -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 15.0.0 -v -fcoverage-compilation-dir=/Users/chris/dev/emsdk -resource-dir /Users/chris/dev/emsdk/upstream/lib/clang/15.0.0 -internal-isystem /usr/local/include -internal-isystem /Users/chris/dev/emsdk/upstream/lib/clang/15.0.0/include -internal-externc-isystem /usr/include -fdebug-compilation-dir=/Users/chris/dev/emsdk -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /var/folders/pz/44zmfmfd26l657xjmcbfr0_00000gn/T/main-2254b8.o -x c main.c
clang -cc1 version 15.0.0 based upon LLVM 15.0.0git default target x86_64-apple-darwin21.3.0
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
/Users/chris/dev/emsdk/upstream/lib/clang/15.0.0/include
/System/Library/Frameworks (framework directory)
/Library/Frameworks (framework directory)
End of search list.
"/usr/bin/ld" -dynamic -arch x86_64 -macosx_version_min 12.0.0 -o a.out /var/folders/pz/44zmfmfd26l657xjmcbfr0_00000gn/T/main-2254b8.o -lSystem
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
Providing an extra search path seems to fix the issue:
chris@localhost emsdk % LIBRARY_PATH="/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib" /Users/chris/dev/emsdk/upstream/bin/clang main.c
chris@localhost emsdk %
Just to note. I managed to build the project I was trying to build. But as well as the above LIBRARY_PATH environment variable, I had to (perhaps unsurprisingly) change the HOST_C_FLAGS to: -W -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/ so that the host compiler could find various standard library headers.
The version of clang provided by emsdk is only really designed to build for the emscripten target. If you want to try to build host executables with the emsdk version of clang you are welcome to try but its not setup for that (as you have discovered). Its not surprising that you would have to inject inject header and library paths as you have done.
Hmm, ok that sort of makes sense. Except that emconfigure and emmake are attempting to make building well behaved projects easy, which it is not doing well by setting HOST_CC to a non-functioning compiler. The project I'm building compiles and runs some host binaries using HOST_CC as an intermediate build step. If this is not the intent behind emconfigure setting HOST_CC, then what is the intent, and why is it set?
Oh I see... that is arguably a (different) bug. Maybe it doesn't make sense for us to set HOST_CC in that way... for example on windows it could be that one wants HOST_CC to be cl.exe, no?
Yeah I'm not sure the correct solution. Ideally HOST_CC would be set to the system's installed compiler I guess? But that requires detecting it. I note that the project I'm building (GMP, https://gmplib.org/ ) builds just fine if I emit the HOST_CC env variable, so I guess it's not really using the host compiler (maybe it needs it for another target that I'm not building I guess). And since no-one else has hit this I guess maybe I can't class it as "emconfigure should just work for this case" :-).
For my use case I think I'll just patch GMP's build stuff in my build script.
What happens if we simply don't set HOST_CC at all? Will it get auto-detected?
I also came across this on macOS15 and... also trying to build gmp (namely https://ftp.gnu.org/gnu/gmp/gmp-6.2.1.tar.bz2).
What happens if we simply don't set HOST_CC at all? Will it get auto-detected?
@sbc100 it will fallback to CC, which is going to be emcc, so it will compile the tools it needs as wasm and will result in lots of obscure errors, while it will start compiling:
./gen-fac 32 0 >fac_table.h || (rm -f fac_table.h; exit
/bin/sh: ./gen-fac: Permission denied
Setting HOST_CC didn't worked for me, as emconfigure overrides it.
What did helped is patching lines below
https://github.com/emscripten-core/emscripten/blob/00c5c7d93f6c898cb4634299164fc5f8b87e10bf/tools/building.py#L62-L63
With:
if 'HOST_CC' not in env:
env['HOST_CC'] = CLANG_CC
if "HOST_CXX" not in env:
env['HOST_CXX'] = CLANG_CXX
The final command I've used for compilation is
HOST_CC=clang emconfigure ../configure --prefix /path/to//gmp-install/ --disable-assembly
Ideally, it should search for the compiler in PATH and set it as HOST_XXX.
Maybe something as simple as checking with shutil.which compilers in order gcc, cc, cl, clang (it's the order that autoconf is using) and setting it to HOST_XXX could be a general solution for this?
Another thing I've stumbled upon compiling gmp is issue with emcc -c conftest.s - basically at some step autoconf is checking compiler whether it's capable of compiling assembly and it's trying to figure what assembly suffix to use (I'll attach piece of code from configure at the end) and with one type of labels it just fails and with other it crashes, so autoconf conclusion is that it fails to Cannot determine label suffix.
Fortunately in gmp there was --disable-assembly option that allows skipping those tests.
That's all probably very expected, since emcc is not intended to compile non-wasm assembly. Just wanted to let know, if anyone else will stumble upon this.
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for assembler label suffix" >&5
$as_echo_n "checking for assembler label suffix... " >&6; }
if ${gmp_cv_asm_label_suffix+:} false; then :
$as_echo_n "(cached) " >&6
else
gmp_cv_asm_label_suffix=unknown
for i in "" ":"; do
echo "trying $i" >&5
cat >conftest.s <<EOF
$gmp_cv_asm_text
somelabel$i
EOF
gmp_assemble="$CCAS $CFLAGS $CPPFLAGS conftest.s >conftest.out 2>&1"
if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$gmp_assemble\""; } >&5
(eval $gmp_assemble) 2>&5
ac_status=$?
$as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
test $ac_status = 0; }; then
cat conftest.out >&5
gmp_cv_asm_label_suffix=$i
rm -f conftest*
break
else
cat conftest.out >&5
echo "configure: failed program was:" >&5
cat conftest.s >&5
cat conftest.out >&5
fi
rm -f conftest*
done
if test "$gmp_cv_asm_label_suffix" = "unknown"; then
as_fn_error $? "Cannot determine label suffix" "$LINENO" 5
fi