Linking Time

[Lokathor]

There was a reddit thread I just saw in the rust_gamedev group, and many people were reporting specifically that link times were killing the development flow.

I figured it needed a tracker/complaint issue here.

---

UPDATE:

@MaulingMonkey advised that lld-link.exe is already a working drop-in replacement on windows that will generally Just Work(tm) with the MSVC toolchain. It's not default but you can configure it to be used while we wait for the default compiler selection to catch up.

You can pass a compiler arg to use the linker:

-Clinker="C:\Program Files\LLVM\bin\lld-link.exe"

Or you can set a config setting in your cargo config file (eg: C:\Users\<username>\.cargo\config):

[target.x86_64-pc-windows-msvc]
linker = "C:\\Program Files\\LLVM\\bin\\lld-link.exe"

It's a little less of a sure thing with the MingW toolchain, but you shouldn't be using that toolchain anyway, so whatever. This should probably work though:

[target.x86_64-pc-windows-gnu]
linker = "C:\\Program Files\\LLVM\\bin\\ld64.lld.exe"
rustflags = ["-Clink-args=-arch x86_64"]

Of course you'll need to install LLVM for this to work: http://releases.llvm.org/download.html

Naturally you already had LLVM on your dev machine because you need it for bindgen, right? ;3

and of course, edit the paths above to whatever folder you install LLVM to if you use a non-default directory.

• Note: As with any other .cargo/config setting, you can also apply this on a per project basis by placing a config file into [project_folder]/.cargo/config, however I don't personally advise this because it makes the project less portable. While the target triple will change from machine to machine, if another windows dev without the LLVM linker tries to build it on their machine they'll suddenly get a build error.

• Note: Using lld-link.exe seems to not work with miri,

[est31]

Personally, I use two workarounds for this:

• First, I use lld instead of GNU bfd. lld is usually faster and it was faster in my own benchmarks too. It also supports threads.
• Second, sometimes I disable debuginfo. A large part of linking time is spent in copying debuginfo over. This problem is so enormous that sometimes debug builds take longer for me than release builds. A remedy is disabling debug info and only turning it on when I want to actually debug a crash/panic/etc. I haven't tested it but I think the profile-overrides feature will help with this as well: using it you can turn off debuginfo for your dependencies and only have debuginfo in your own code, which is where most of your debugging happens :). This issue might also help with speeding up debuginfo related linking time.

This is the ~/.cargo/config snippet I have to use lld:

# This speeds up small changes dramatically.
# With lld enabled, doing a small change and then
# recompiling takes 0,789s in total on a project of mine.
# Without lld, the same change takes 1,842s.
[target.x86_64-unknown-linux-gnu]
linker = "/usr/bin/clang"
rustflags = ["-Clink-arg=-fuse-ld=lld"]

Disabling debuginfo on a per-project basis is easy as well: example commit.

[zicklag]

I have experienced the long linking times too and also ended up using LLD. To install LLD I just renamed it to ld and put it in ~/.local/bin/.

I hadn't heard about the debuginfo stuff. That is a really useful tip! :smiley: Compile times ( more specifically re-compiling times ) are the single largest problem that I have with using Rust in general so any tips and tricks to speed that up during development would be great to have consolidated somewhere.

[Wodann]

I am working on hot reloading of compartmentalised shared libraries. At the moment shared libraries are still generated per crate, but it might even be doable to split this up on module-basis in the future. Apart from improving developer efficiency, it also has the potential of reducing compile times. This approach is similar to the one described by Embark Studios (https://github.com/EmbarkStudios/rust-ecosystem/issues/13)

For the current state of the project, have a look at this Reddit thread. In the near future, I will upload the latest version of the source code here.

[hadronized]

I am working on hot reloading of compartmentalised shared libraries. At the moment shared libraries are still generated per crate, but it might even be doable to split this up on module-basis in the future. Apart from improving developer efficiency, it also has the potential of reducing compile times. This approach is similar to the one described by Embark Studios (EmbarkStudios/rust-ecosystem#13)

For the current state of the project, have a look at this Reddit thread. In the near future, I will upload the latest version of the source code here.

For development I’ve always considered doing it, but for release, because of Rust lacking a proper ABI, it might never work. 😢

[Lokathor]

Note: I've updated the top post of this issue with a note on how to configure lld-link.exe for usage (both Stable or Nightly), while we wait for the default compiler to catch up. Not 100% assured to work with all flags and stuff, but for basic projects it should be a drop-in replacement.

[MaulingMonkey]

This can really really help for Android builds as well. On professional C++ codebases I've previously cut link times from >1 minute to ~10 seconds - per project (many) x arch (4 - x86, x64, arm, arm64) x build (many) combination - by just using -fuse-ld=gold, and lld might be even faster. You can also put those .cargo/config files in your actual workspace root which can be easier to share.

For the x86_64-pc-windows-gnu toolchain I should note that I did encounter issues linking the rust stdlib, so that probably doesn't work quite as nicely...

[AngelOnFira]

Hey all, not much for advice but for the Veloren project we're doing some unique stuff:

• Disabled debuginfo, working on our own better logging system
• We're using the gold linker. Here is what our config file looks like:

[target.x86_64-unknown-linux-gnu]                                                                   
rustflags = [                                                                                       
    "-C", "link-arg=-fuse-ld=gold",
]

I think there is a reason we're using gold over lld, but I'm not too sure. We were going to switch from vanilla to lld, but gold was the one we chose.

I don't have any numbers on our CI improvements, as our runners are each custom and unique.

Also, here is our dev profile:

# default profile for devs, fast to compile, okay enough to run, no debug information
[profile.dev]
opt-level = 2
overflow-checks = true
debug-assertions = true
panic = "abort"
debug = false
codegen-units = 8
lto = false
incremental = true
# All dependencies (but not this crate itself)
[profile.dev.overrides."*"]
opt-level = 3
[profile.dev.overrides."veloren-common"]
opt-level = 2
[profile.dev.overrides."veloren-client"]
opt-level = 2
[profile.dev.overrides."veloren-chat-cli"]
opt-level = 2
[profile.dev.overrides."veloren-server"]
opt-level = 2
[profile.dev.overrides."veloren-server-cli"]
opt-level = 2
[profile.dev.overrides."veloren-voxygen"]
opt-level = 2
[profile.dev.overrides."veloren-world"]
opt-level = 2

[Lokathor]

Update: Seems to play badly with miri (noted in the OP)

[asv7c2]

Looks like ld64.lld linker cannot link stdlib on target.x86_64-pc-windows-gnu