`RUSTFLAGS` support

[InBetweenNames]

The LLVM framework has its own way of handling LTO, using ThinLTO or otherwise. We should investigate how to add that into GentooLTO using the RUSTFLAGS environment variable. I'm thinking we should use ThinLTO by default, and fallback to plain LTO if that fails. I think another flag, RUSTLINKFLAGS, may be of use for these packages.

This should help with the rust related packages we've been seeing more often here. Unfortunately, it's not possible to perform interprocedural optimizations using LTO across GCC and LLVM boundaries. If a GCC frontend for rust ever emerges, then we'd be able to optimize across rust, C, and C++ boundaries. Otherwise, you'd have to use clang system-wide to achieve that result, as in issue #146. This could be particularly good for www-client/firefox, for example.

[InBetweenNames]

Now that I think about it, we can handle Fortran right now. I'm pretty sure we can set FFLAGS and FCFLAGS to CFLAGS without any issue and gain LTO that way.

[InBetweenNames]

Additional reading: https://github.com/rust-lang/rust/issues/48518

Interestingly, ThinLTO leaves out certain optimizations that monolithic LTO enables. I'm experimenting with monolithic LTO currently.

[InBetweenNames]

To anyone wanting to try out rust LTO. For ThinLTO:

RUSTFLAGS="-C codegen-units=${NTHREADS} -Z thinlto"

For monolithic LTO:

RUSTFLAGS="-C lto=fat -C codegen-units=1"

[InBetweenNames]

Ahh yes, I moved a little too fast on this one. It turns out that F*FLAGS for Fortran are automatically set by package.cflags to CFLAGS. Furthermore, OBJ*FLAGS are also set to CFLAGS in the few ebuilds those variables are actually used in. I will update the documentation accordingly. No breaking changes at least.

[ionenwks]

Not a LTO issue but on a quick side-note, can't say I know much about rust but I had been building firefox with RUSTFLAGS="-Ctarget-cpu=native -Copt-level=3" for a while, but since firefox 65.0 it segfaults unless I drop to -Copt-level=2. Took me a bit to figure out it was that since it had kinda slipped from my mind I set it. Edit: Now that I think about it, may have been a side-effect of upgrading to rust 1.32 at the time, didn't experiment enough. That aside, it'd probably be good to have some default RUSTFLAGS with at least -Ctarget-cpu=native in make.conf

[ghost]

@InBetweenNames

To anyone wanting to try out rust LTO. For ThinLTO:

RUSTFLAGS="-C codegen-units=${NTHREADS} -Z thinlto"

The -Z flag doesn't work (anymore?), is this because I'm not on a nightly rust build? I did some research on my own and these are the flags I came up with:

-Copt-level=3 -Ctarget-cpu=${MARCH} -Ccodegen-units=${THREADS} -Clto=thin -Clinker-plugin-lto

(MARCH is set to the processor architecture, (n)threads equals the thread count)

I'm having issues compiling with these flags. Rust complains that options '-C embed-bitcode=no' and '-C lto' are incompatible. This error seems to contradict the relevant documentation (see note); since I've enabled linker-plugin-lto, I shouldn't have to embed bitcode additionally and shave off some compilation time. Am I understanding this correctly? Where is this error coming from?

[Hello71]

that doesn't work because cargo has its own idea of whether lto is active.

bashrc:

post_src_prepare() {
    if ! [[ -d ${CARGO_HOME} ]]; then
        local CARGO_HOME="$HOME/.cargo"
        mkdir ${CARGO_HOME}
    fi
    # cargo.eclass uses ${ECARGO_HOME}/config, not config.toml
    cat >> ${CARGO_HOME}/config << EOF
[profile.release]
lto = "thin"
EOF
}

make.conf:

RUSTFLAGS="-C opt-level=3 -C linker=clang -C target-cpu=native -C linker-plugin-lto -C link-arg=-fuse-ld=lld"

notes:

• -C linker-plugin-lto requires -C link-arg=-fuse-ld=lld and linker = "clang"
• -C opt-level=3 is default in cargo, but firefox unsets it for nonsense reasons

[starquake]

So do I understand this correctly and is RUSTFLAGS="-C opt-level=3 -C linker=clang -C target-cpu=native -C linker-plugin-lto -C link-arg=-fuse-ld=lld" the same as lto = "thin" in Cargo.tml?

EDIT: Compilation failed so I guess it is not

[Hello71]

no. as I said, you must enable lto in both rustc and cargo, if cargo is in use.

[starquake]

Okay thanks, Rust stuff is all quite new to me.

[jiblime]

that doesn't work because cargo has its own idea of whether lto is active.

bashrc:

post_src_prepare() {

Is there documentation on post_ phases? I was surprised to see it work without using BashrcdPhase to call the function.

[telans]

I haven't heard of it either. Looks like they're called hook functions: https://dev.gentoo.org/~zmedico/portage/doc/portage.html#config-bashrc-ebuild-phase-hooks

It's only mentioned in passing in the bashrc wiki page: https://wiki.gentoo.org/wiki//etc/portage/bashrc

[Hello71]

it could probably use BashrcdPhase too, like BashrcdPhase prepare set_cargo_lto but this way works without bashrcd installed. note that it must run between cargo_src_unpack and cargo_src_compile for cargo.eclass ebuilds.

[perfect7gentleman]

@Hello71, does "-C linker-plugin-lto" still works in your system configuration as in my system "-C linker-plugin-lto" does't work anymore ?

[Hello71]

RUSTFLAGS is fragile as a system-wide config, since stacking is handled inconsistently by each package (usually overridden). Additionally, linker-plugin-lto requires clang LTO to be used; LLVM LTO is not compatible with gcc LTO.