homebrew-core icon indicating copy to clipboard operation
homebrew-core copied to clipboard

Install wrapper script to set `LD_RUN_PATH` pointing to homebrew lib

Open straight-shoota opened this issue 1 year ago • 13 comments

The Crystal compiler requires a linker and by default it just picks up whatever gcc it finds in PATH. This causes issues with discovering libraries installed via homebrew. We previously tried to solve this with #162182, but that doesn't work. Now the Crystal compiler has gained support for a configuration value CRYSTAL_CONFIG_CC which allows changing the baked-in default linker path (https://github.com/crystal-lang/crystal/pull/14318). So we can point it directly to the gcc from homebrew. And this resolves all issues with dependency discovery. The compiler change is not yet released, so it only works with --HEAD.

It's still possible to specify a non-default linker via the environment variable CC at runtime, of course.

This effectively reverts #162182 and replaces it with an alternative which actually works.

straight-shoota avatar Mar 13 '24 17:03 straight-shoota

Do we need this on macOS? Original issue sounds like we only need to add this dependency on_linux. crystal run seems to work fine on macOS

Also feels slightly weird installing both LLVM and GCC here, but maybe it's necessary.

Bo98 avatar Mar 13 '24 17:03 Bo98

Do we need this on macOS?

I'm not sure and I'm not familiar with the specifics of macOS. However, I would expect the same problem could apply there as well. If cc resolves to a linker that is not installed via homebrew, it probably might not find the libraries installed only in homebrew?

crystal run seems to work fine on macOS

It used to work on Linux as well, but now it doesn't. And it's not entirely clear what has changed. We should probably be suprised that it worked before 🤷 Note that this is not an issue if the required libraries are available in the system configuration. Then the linker will just pick those and you'll never notice they aren't from homebrew (unless you look into it).

Also feels slightly weird installing both LLVM and GCC here, but maybe it's necessary.

LLVM is only a lib dependency of the compiler. We could trim the dependency down to just libllvm. GCC is used solely as a linker. Technically, GCC shouldn't be necessary and we should only need a pure linker (such as ld). But currently the Crystal compiler relies on some features that GCC sprinkles on top. This might change in the future. (https://github.com/crystal-lang/crystal/issues/14332). It should also be possible to use clang as linker, but gcc is typically the default and I'm sure changing that would lead to many suprises.

straight-shoota avatar Mar 13 '24 18:03 straight-shoota

On macOS, it links correctly with the default system compiler:

$ crystal build test.cr
$ otool -L test
test:
	/opt/homebrew/opt/pcre2/lib/libpcre2-8.0.dylib (compatibility version 12.0.0, current version 12.2.0)
	/opt/homebrew/opt/bdw-gc/lib/libgc.1.dylib (compatibility version 7.0.0, current version 7.3.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)
	/opt/homebrew/opt/libevent/lib/libevent-2.1.7.dylib (compatibility version 8.0.0, current version 8.1.0)
	/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)

this is because on macOS it links with the absolute paths linked in:

$ otool -l test
[...]
Load command 15
          cmd LC_LOAD_DYLIB
      cmdsize 72
         name /opt/homebrew/opt/bdw-gc/lib/libgc.1.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 7.3.0
compatibility version 7.0.0
[...]

so it doesn't use an rpath system like Linux does.

This might change in the future. (https://github.com/crystal-lang/crystal/issues/14332).

It's worth noting that if the future direction is linker-level support then our GCC hack won't kick in and you'll need to implement an rpath system for that.

Bo98 avatar Mar 13 '24 18:03 Bo98

I'm happy to opt-out on macOS. IIUC gcc itself is also only modified on linux.

Not sure if it makes sense to keep the gcc dependency though. Crystal needs a C compiler to use for linking. By default it uses cc (or $CC if set). I suppose the homebrew formula doesn't fill this alias, so the dependency is probably mute if it's not hard-coded? I suppose that could be a reason to keep CRYSTAL_CONFIG_CC on macOS, though. 🤷

Even with gcc dependency, out-of-the-box experience with the crystal formula seems pretty poor, though. I tried brew install crystal on a blank install of Debian linux, with a blank homebrew; it fails miserably.

straight-shoota avatar Mar 13 '24 21:03 straight-shoota

It might make sense to add the dependency in the formula without restrictions (and be sure to use it, also in macOS). In any case, we can first try this for the Linux version and then see if we really want to bother the macOS version with it.

beta-ziliani avatar Apr 08 '24 20:04 beta-ziliani

Putting the gcc dependency in on_linux causes a commit style error:

Formulae in homebrew/core should not have a Linux-only dependency on GCC.

@Bo98 Any advice on what I should do? I could revert to f44c79f47dd388f3be5205c082fc1ac53f6897d9 which seemed fine. That adds gcc as a dependency on all platforms, which isn't wrong (you need a C compiler to use Crystal).

straight-shoota avatar Apr 09 '24 12:04 straight-shoota

GCC is the default on Linux so no need to specify a dependency unless you need a specific version

SMillerDev avatar Apr 09 '24 15:04 SMillerDev

@SMillerDev I tried to remove the gcc dependency entirely, but then Formula["gcc"] is nil and we can't get the path to the C compiler from there. How could this work?

straight-shoota avatar Apr 09 '24 17:04 straight-shoota

I'm temporarily closing this PR to let autobump update the formula.

straight-shoota avatar Apr 09 '24 18:04 straight-shoota

Audit seems to be unhappy with having gcc as a dependency only on Linux, so I guess it should really just be a general dependency.

straight-shoota avatar Apr 24 '24 20:04 straight-shoota

I'm temporarily closing this PR to let autobump update the formula. (again)

straight-shoota avatar Jun 05 '24 18:06 straight-shoota

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

github-actions[bot] avatar Jul 07 '24 14:07 github-actions[bot]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

github-actions[bot] avatar Aug 17 '24 00:08 github-actions[bot]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

github-actions[bot] avatar Sep 11 '24 22:09 github-actions[bot]

:warning: @carlocab bottle publish failed.

github-actions[bot] avatar Sep 19 '24 04:09 github-actions[bot]

Blocked by #181351.

carlocab avatar Sep 19 '24 06:09 carlocab

Merged in 085c8863346915267ba07e89d6148ca491ed3ef1. Thanks @straight-shoota!

carlocab avatar Sep 23 '24 18:09 carlocab