Support using system clang

[Akaricchi]

I'd be nice to have an option to use the clang and LLVM tools provided by my distro with this toolchain. Do you think that'd be feasible?

[mstorsjo]

It's been a design criteria for me that this should be possible; I don't apply any custom modifications to the clang/llvm/lld build (but all tweaks are handled outside of that) to make sure that any vanilla clang/lld should work.

It's not automated and scripted, and I haven't tested doing it myself, but here are a few things that come to mind:

Ideally you would use a version similar to the one I have pinned here (currently the 9.0.0 release). In practice, you can manage with older versions, with some restrictions

• LLD 6.0 is the first one to have the MinGW frontend driver, and it's been gradually improved ever since, so this is the absolute minimum (and I don't remember off-hand if there's any critical bits after this, or if it's just nice-to-have things)
• LLVM 9.0 fixed a bug with dwarf unwind info generation (for exeption handling) on i686. (Before the bug was fixed, I overrided the default and used sjlj exception handling instead, but the current version of the wrapper scripts assume that you can use dwarf for exceptions on i686.)
• llvm-objcopy/llvm-strip didn't support COFF until in 9.0

To try building the toolchain with an external build of clang/lld/llvm, follow the steps of build-all.sh, but skip the build-llvm.sh and strip-llvm.sh steps, and instead do this:

Create the target toolchain directory, and copy/symlink/hardlink the following necessary executables into it: clang, ld.lld, llvm-ar, llvm-ranlib, llvm-nm, llvm-objcopy, llvm-strip, llvm-rc, llvm-cvtres. The install-wrappers.sh script also tries to make symlinks for llvm-addr2line and llvm-strings, but I'm fairly sure they aren't actually used by the toolchain build.

For most tools, symlinking should be fine. Clang is an exception. Clang uses its own resource directory, located at <clang-executable>/../lib/clang/<clang-version>. If you use a symlink, clang resolves this path relative to the actual executable, not to the link.

If you used a symlink, you will need to manually copy <new-toolchain>/lib/clang/<clang-version/lib/windows into the original clang binary's resource directory (which might be in /usr, i.e. not recommended to manually edit) after running build-compiler-rt.sh. (This is called twice; first once to build compiler builtins, then once later to build sanitizers; you need to copy the newly built files after the other step as well if you want to use sanitizers.) You could of course also try to symlink the resource dir, but then build-compiler-rt.sh probably doesn't have write access to it.

If you did not use a symlink, but hardlinked/copied the clang binary, then copy the include subdirectory from the clang resource dir to the matching directory in the new toolchain.

After that, running the following steps from build-all.sh should work fine. As llvm nowadays exists in a large monorepo containing all the related projects, you'll end up cloning this huge repo (around 1 GB for the git data, and another 1 GB for the checked out files) even if you only are building the runtimes (compiler-rt, libunwind, libcxxabi, libcxx), unfortunately.

[nephatrine]

I'm trying this out how you've outlined here, but running into some issues when it's trying to build-mingw-w64.sh. Looks like there's a lot of complaining about redefinitions between the system clang includes (/usr/lib/clang/9.0.0/include/intrin.h) and mingw's ($TOOLCHAIN_PREFIX/i686-w64-mingw32/include/intrin.h).

Everything seems to work fine if I building my own llvm as in your Dockerfile or build-all.sh, but I'd prefer to reuse system llvm since that takes a really long time and duplicates a lot of stuff already installed.

This works:

./build-llvm.sh $TOOLCHAIN_PREFIX
./install-wrappers.sh $TOOLCHAIN_PREFIX
./build-mingw-w64.sh $TOOLCHAIN_PREFIX --with-default-msvcrt=ucrt

This does not:

mkdir -p $TOOLCHAIN_PREFIX/bin/
cp -nrs /usr/lib/llvm-9/bin/* $TOOLCHAIN_PREFIX/bin/
./install-wrappers.sh $TOOLCHAIN_PREFIX
./build-mingw-w64.sh $TOOLCHAIN_PREFIX --with-default-msvcrt=ucrt

I'm using the llvm-9.0 branch since that's the system version of llvm, but I get the same with master.

Here's an excerpt from where it starts to fail:

mv -f misc/.deps/lib32_libmingwex_a-btowc.Tpo misc/.deps/lib32_libmingwex_a-btowc.Po
i686-w64-mingw32-gcc -DHAVE_CONFIG_H -I. -I..  -m32 -I../include -I/llvm-mingw/i686-w64-mingw32/include  -pipe -std=gnu99 -D_CRTBLD -D_WIN32_WINNT=0x0f00 -D__MSVCRT_VERSION__=0x700 -Wall -Wextra -Wformat -Wstrict-aliasing -Wshadow -Wpacked -Winline -Wimplicit-function-declaration -Wmissing-noreturn -Wmissing-prototypes -g -O2 -MT misc/lib32_libmingwex_a-dirent.o -MD -MP -MF misc/.deps/lib32_libmingwex_a-dirent.Tpo -c -o misc/lib32_libmingwex_a-dirent.o `test -f 'misc/dirent.c' || echo '../'`misc/dirent.c
In file included from ../math/abs64.c:1:
In file included from /usr/lib/llvm-9/lib/clang/9.0.0/include/intrin.h:12:                                                                                   
/llvm-mingw/i686-w64-mingw32/include/intrin.h:87:36: error: typedef redefinition with different types ('union __m64' vs '__attribute__((__vector_size__(1 * sizeof(long long)))) long long' (vector of 1 'long long' value))                                                                                              
typedef union __m64 { char v[7]; } __m64;                                                                                                                    
                                   ^                                                                                                                         
/usr/lib/llvm-9/lib/clang/9.0.0/include/mmintrin.h:13:19: note: previous definition is here                                                                  
typedef long long __m64 __attribute__((__vector_size__(8), __aligned__(8)));

[mstorsjo]

Hmm, so,

I'm trying this out how you've outlined here, but running into some issues when it's trying to build-mingw-w64.sh. Looks like there's a lot of complaining about redefinitions between the system clang includes (/usr/lib/clang/9.0.0/include/intrin.h) and mingw's ($TOOLCHAIN_PREFIX/i686-w64-mingw32/include/intrin.h).

Everything seems to work fine if I building my own llvm as in your Dockerfile or build-all.sh, but I'd prefer to reuse system llvm since that takes a really long time and duplicates a lot of stuff already installed.

This works:

./build-llvm.sh $TOOLCHAIN_PREFIX
./install-wrappers.sh $TOOLCHAIN_PREFIX
./build-mingw-w64.sh $TOOLCHAIN_PREFIX --with-default-msvcrt=ucrt

This does not:

mkdir -p $TOOLCHAIN_PREFIX/bin/
cp -nrs /usr/lib/llvm-9/bin/* $TOOLCHAIN_PREFIX/bin/
./install-wrappers.sh $TOOLCHAIN_PREFIX
./build-mingw-w64.sh $TOOLCHAIN_PREFIX --with-default-msvcrt=ucrt

I'm using the llvm-9.0 branch since that's the system version of llvm, but I get the same with master.

Here's an excerpt from where it starts to fail:

mv -f misc/.deps/lib32_libmingwex_a-btowc.Tpo misc/.deps/lib32_libmingwex_a-btowc.Po
i686-w64-mingw32-gcc -DHAVE_CONFIG_H -I. -I..  -m32 -I../include -I/llvm-mingw/i686-w64-mingw32/include  -pipe -std=gnu99 -D_CRTBLD -D_WIN32_WINNT=0x0f00 -D__MSVCRT_VERSION__=0x700 -Wall -Wextra -Wformat -Wstrict-aliasing -Wshadow -Wpacked -Winline -Wimplicit-function-declaration -Wmissing-noreturn -Wmissing-prototypes -g -O2 -MT misc/lib32_libmingwex_a-dirent.o -MD -MP -MF misc/.deps/lib32_libmingwex_a-dirent.Tpo -c -o misc/lib32_libmingwex_a-dirent.o `test -f 'misc/dirent.c' || echo '../'`misc/dirent.c
In file included from ../math/abs64.c:1:
In file included from /usr/lib/llvm-9/lib/clang/9.0.0/include/intrin.h:12:                                                                                   
/llvm-mingw/i686-w64-mingw32/include/intrin.h:87:36: error: typedef redefinition with different types ('union __m64' vs '__attribute__((__vector_size__(1 * sizeof(long long)))) long long' (vector of 1 'long long' value))                                                                                              
typedef union __m64 { char v[7]; } __m64;                                                                                                                    
                                   ^                                                                                                                         
/usr/lib/llvm-9/lib/clang/9.0.0/include/mmintrin.h:13:19: note: previous definition is here                                                                  
typedef long long __m64 __attribute__((__vector_size__(8), __aligned__(8)));

So, if looking at mingw-w64's intrin.h, line 86-88, we have this:

#ifndef __MMX__
typedef union __m64 { char v[7]; } __m64;
#endif

So it seems like your system's clang 9 doesn't define __MMX__ by default when targeting i686-w64-mingw32. If you have both setups (working and broken), you can do e.g. i686-w64-mingw32-gcc -E -dM - < /dev/null with both of them and compare the output. If that shows a difference, you can also simplify things and take the wrapper out of the question and reproduce it with /usr/lib/llvm-9/bin/clang -target i686-w64-mingw32 -E -dM - < /dev/null.

If there's differences between what they produce (which it would seem), it could be that your system clang is configured with some non-default options to make it not enable MMX by default. In general that should be fine, but mingw-w64's intrin.h header and compatibility with the GCC and clang built-in headers is a total rat's nest, and this situation hasn't come up in any other testing yet.

I think one fix might be to change mingw-w64's intrin.h header like this:

diff --git a/mingw-w64-headers/crt/intrin.h b/mingw-w64-headers/crt/intrin.h
index cf284c308..e8a5e7bd9 100644
--- a/mingw-w64-headers/crt/intrin.h
+++ b/mingw-w64-headers/crt/intrin.h
@@ -40,7 +40,7 @@
 #include <stddef.h>
 #include <psdk_inc/intrin-impl.h>
 
-#if __MINGW_GNUC_PREREQ(4, 9)
+#if __MINGW_GNUC_PREREQ(4, 9) || defined(__clang__)
 #define __MINGW_FORCE_SYS_INTRINS
 #endif
 

(Clang normally presents itself as gcc 4.2, as there are certain specific gcc 4.3 features that it doesn't support.)

I have no idea if that's a good fix, or if that just will turn up a bunch of other problems elsewhere though.

If your system clang happens to be one that disables mmx/sse by default contrary to vanilla clang, you might be able to get back to the default with something like this (untested):

diff --git a/wrappers/clang-target-wrapper.sh b/wrappers/clang-target-wrapper.sh
index a6a4140..0d5f129 100755
--- a/wrappers/clang-target-wrapper.sh
+++ b/wrappers/clang-target-wrapper.sh
@@ -38,6 +38,7 @@ esac
 case $ARCH in
 i686)
     # Dwarf is the default for i686.
+    FLAGS="$FLAGS -mmmx -msse -msse2"
     ;;
 x86_64)
     # SEH is the default for x86_64.

[nephatrine]

Yes, that turned out to be exactly it. It appears that Ubuntu has it configured to tune for i686/pentium pro and so does not enable mmx/sse by default. Adding flags in the clang-target-wrapper as you specified solved it for me. Thanks for pointing out where the issue was.