Handling LLVM builds with LTOed objects

[carlocab]

Shall we continue our discussion from Homebrew/discussions#3666 here?

Here's a summary of alternatives discussed so far:

1. Revert the change in Homebrew. This may be essentially equivalent to not supporting any LLVM builds configured with LLVM_ENABLE_LTO=ON on macOS.
2. Link with the libLLVM shared library instead.
3. Require extra configuration from Homebrew users (e.g. setting C[XX]FLAGS)
4. Somehow detecting an LTOed build and configuring it accordingly.

2 and 3 seem like the most attractive alternatives now, but it may still be possible to do 1 (depending on how other users receive the changes wrt LTO), or even 4 if there is a neat enough way to do it.

[banach-space]

Thanks for opening this issue and for all the help resolving this so far!

I am currently experimenting with 2 as the least intrusive approach from the llvm-tutor perspective. Currently waiting for the CI to confirm that this will work (my earlier run failed, but that should be fixed now). It will mean building and linking a much larger library than previously (i.e. libLLVM), but hopefully that's not going to be prohibitive for folks.

Yes, 3 would also be good. I'm not sure how to achieve 4 🤔 . As for 1, I would welcome it, but wouldn't expect Homebrew to make such change for llvm-tutor alone :)

[carlocab]

Another thought: maybe it is possible to do some combination of 2 and 4.

In particular, we can test whether it is possible to link only with the static libraries (e.g. with CMake's try_compile). If that succeeds, then we proceed as before. If that does not, we fall back to linking with the libLLVM shared library.

That way you link with the static libraries when you're able to, but it will "just work" even when you're not.

This might even have the benefit of handling any other LLVM builds that happen to have been configured with LLVM_ENABLE_LTO, including those not from Homebrew. How does that sound?

[banach-space]

You are quite resourceful :)

This might even have the benefit of handling any other LLVM builds that happen to have been configured with LLVM_ENABLE_LTO, including those not from Homebrew.

Very good point!

How does that sound?

I like the idea - it is now being tested (commit) :)

My only concern is that it complicates the CMake configuration a bit. llvm-tutor is intended for people with potentially very little CMake experience. Also, llvm-tutor strives to demonstrate how easy it is to build stuff based on LLVM. So "less is more". But then expecting that every installation (on various platforms) of LLVM will be identical is unrealistic in the long term.

[banach-space]

I've just updated the release_15 branch. I'm mostly happy with it - I used try_compile).

Bbefore merging I still want to:

• let the CI verify it,
• double-check the documentation

I've added you as a co-author - without your help I would still be stuck here. Should I add an e-mail address there?

[carlocab]

No problem; thanks for the acknowledgment. You can add an e-mail address if you wish, here's an example of my usual Co-authored-by: trailer: https://github.com/Homebrew/homebrew-core/commit/6838f14930f89688890c4f430d849abcbfd61a8e

Or you can use [email protected].

[banach-space]

I've just merged the LLVM 15 update 🥳 .

They key CI jobs (Ubuntu + Darwin) are 🟢 . A handful of CI jobs are failing, but these are the docker jobs for other OSes (Fedora and Arch Linux) that normally take a bit longer to update their LLVM packages. I will be looking into these in the coming days.

Thank you so much for all your help - that's much appreciated!

[carlocab]

Glad this is working!

I submitted a patch upstream that might help you significantly simplify things here: https://reviews.llvm.org/D134936

[banach-space]

Oh, nice! I won't be able to use it until LLVM 16, but will have something to look forward to in the meantime :)

[carlocab]

I abandoned my patch, but you shouldn't need it anymore, since I recently fixed Homebrew LLVM to enable linkage with the static libraries to just work. I did this via the trick borrowed from Fedora.

Users who encounter bitcode-related errors should do

brew update && brew reinstall llvm

[carlocab]

Oh, and thanks again for bearing with the breakage and for your flexibility in dealing with it!

[banach-space]

Thanks for the heads-up, I'll tweak llvm-tutor over the weekend.

Oh, and thanks again for bearing with the breakage and for your flexibility in dealing with it!

More than happy to - it is important to support each other! :)