rust-lang
sudo cargo install recompile everything after cargo build --release
Problem
Users may want to install a rust binary crate to /usr/local/bin, not to $HOME/.cargo/bin, and this can be archived by --root flag of cargo install.
And normal users don't have the permission to write /usr/local/bin directly, so they have to run
sudo cargo install --root /usr/local instead.
The bug is here: when cargo is running as another user, it will recompile everything due to the environment change.
Steps
> #unset anything tracked by cargo
> $HOME/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/bin/cargo new hello
Created binary (application) `hello` package
> $HOME/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/bin/cargo build --release
Compiling hello v0.1.0 (/tmp/hello)
Finished release [optimized] target(s) in 0.81s
> $HOME/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/bin/cargo install --path ./ --root /tmp/usr
Installing hello v0.1.0 (/tmp/hello)
Finished release [optimized] target(s) in 0.01s
Replacing /tmp/usr/bin/hello
Replaced package `hello v0.1.0 (/tmp/hello)` with `hello v0.1.0 (/tmp/hello)` (executable `hello`)
warning: be sure to add `/tmp/usr/bin` to your PATH to be able to run the installed binaries
> $HOME/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/bin/cargo build --release
Finished release [optimized] target(s) in 0.01s
> sudo $HOME/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/bin/cargo install --path ./ --root /tmp/usr
Installing hello v0.1.0 (/tmp/hello)
Compiling hello v0.1.0 (/tmp/hello)
Finished release [optimized] target(s) in 5.27s
Replacing /tmp/usr/bin/hello
Replaced package `hello v0.1.0 (/tmp/hello)` with `hello v0.1.0 (/tmp/hello)` (executable `hello`)
warning: be sure to add `/tmp/usr/bin` to your PATH to be able to run the installed binaries
( Don't use sudo cargo directly when there is another system wide rust installation )
Possible Solution(s)
cargo should detect this situation.
A fallback solution is adding a new flags like "install-only" that don't recompile no matter the environment is changed or not.
Notes
cmake & ninja won't recompile everything when run as root, so you can use
mkdir build && cd build && cmake .. -GNinja
ninja
sudo ninja install
to install any cmake based project.
I first notice this issue when I compile some rust packages using Portage (gentoo's package manager).
>>> Source unpacked in /tmp/portage/sys-apps/bat-0.20.0/work
>>> Preparing source in /tmp/portage/sys-apps/bat-0.20.0/work/bat-0.20.0 ...
>>> Source prepared.
>>> Configuring source in /tmp/portage/sys-apps/bat-0.20.0/work/bat-0.20.0 ...
>>> Source configured.
>>> Compiling source in /tmp/portage/sys-apps/bat-0.20.0/work/bat-0.20.0 ...
* cargo build --release
Compiling libc v0.2.112
Compiling proc-macro2 v1.0.36
Compiling unicode-xid v0.2.2
Compiling syn v1.0.85
Compiling pkg-config v0.3.24
Compiling autocfg v1.0.1
Compiling serde_derive v1.0.136
Compiling cfg-if v1.0.0
Compiling memchr v2.4.1
Compiling serde v1.0.136
Compiling encoding_index_tests v0.1.4
Compiling regex-syntax v0.6.25
Compiling proc-macro-hack v0.5.19
Compiling lazy_static v1.4.0
Compiling log v0.4.14
Compiling tinyvec_macros v0.1.0
Compiling matches v0.1.9
Compiling unicode-width v0.1.9
Compiling bitflags v1.3.2
Compiling crc32fast v1.3.0
Compiling hashbrown v0.11.2
Compiling ryu v1.0.9
Compiling itoa v0.4.8
Compiling regex-automata v0.1.10
Compiling vec_map v0.8.2
Compiling adler v1.0.2
Compiling fnv v1.0.7
Compiling ansi_term v0.12.1
Compiling same-file v1.0.6
Compiling strsim v0.8.0
Compiling linked-hash-map v0.5.4
Compiling percent-encoding v2.1.0
Compiling safemem v0.3.3
Compiling serde_json v1.0.74
Compiling unicode-bidi v0.3.7
Compiling base64 v0.13.0
Compiling semver v1.0.4
Compiling itoa v1.0.1
Compiling bytemuck v1.7.3
Compiling bugreport v0.4.1
Compiling xml-rs v0.8.4
Compiling once_cell v1.9.0
Compiling std_prelude v0.2.12
Compiling lazycell v1.3.0
Compiling termcolor v1.1.2
Compiling shell-escape v0.1.5
Compiling shell-words v1.0.0
Compiling bytesize v1.1.0
Compiling wild v2.0.4
Compiling encoding-index-singlebyte v1.20141219.5
Compiling encoding-index-simpchinese v1.20141219.5
Compiling encoding-index-korean v1.20141219.5
Compiling encoding-index-japanese v1.20141219.5
Compiling encoding-index-tradchinese v1.20141219.5
Compiling tinyvec v1.5.1
Compiling indexmap v1.7.0
Compiling miniz_oxide v0.4.4
Compiling walkdir v2.3.2
Compiling form_urlencoded v1.0.1
Compiling yaml-rust v0.4.5
Compiling line-wrap v0.1.1
Compiling rgb v0.8.31
Compiling path_abs v0.5.1
Compiling encoding v0.2.33
Compiling unicode-normalization v0.1.19
Compiling ansi_colours v1.1.1
Compiling aho-corasick v0.7.18
Compiling bstr v0.2.17
Compiling content_inspector v0.2.4
Compiling quote v1.0.14
Compiling jobserver v0.1.24
Compiling term_size v0.3.2
Compiling atty v0.2.14
Compiling time v0.3.5
Compiling dirs-sys-next v0.1.2
Compiling terminal_size v0.1.17
Compiling textwrap v0.11.0
Compiling idna v0.2.3
Compiling cc v1.0.72
Compiling dirs-next v2.0.0
Compiling regex v1.5.4
Compiling clap v2.34.0
Compiling url v2.2.2
Compiling globset v0.4.8
Compiling console v0.15.0
Compiling flate2 v1.0.22
Compiling grep-cli v0.1.6
Compiling libz-sys v1.1.3
Compiling onig_sys v69.7.1
Compiling sys-info v0.9.1
Compiling libgit2-sys v0.12.26+1.3.0
Compiling onig v6.3.1
Compiling bat v0.20.0 (/tmp/portage/sys-apps/bat-0.20.0/work/bat-0.20.0)
Compiling git2 v0.13.25
Compiling git-version-macro v0.3.5
Compiling thiserror-impl v1.0.30
Compiling thiserror v1.0.30
Compiling git-version v0.3.5
Compiling bincode v1.3.3
Compiling plist v1.3.1
Compiling clircle v0.3.0
Compiling serde_yaml v0.8.23
Compiling syntect v4.6.0
Finished release [optimized] target(s) in 3m 06s
>>> Source compiled.
>>> Test phase [not enabled]: sys-apps/bat-0.20.0
>>> Install sys-apps/bat-0.20.0 into /tmp/portage/sys-apps/bat-0.20.0/image
* cargo install --path ./ --root /tmp/portage/sys-apps/bat-0.20.0/image/usr
Installing bat v0.20.0 (/tmp/portage/sys-apps/bat-0.20.0/work/bat-0.20.0)
Compiling libc v0.2.112
Compiling syn v1.0.85
Compiling atty v0.2.14
Compiling term_size v0.3.2
Compiling time v0.3.5
Compiling terminal_size v0.1.17
Compiling flate2 v1.0.22
Compiling dirs-sys-next v0.1.2
Compiling jobserver v0.1.24
Compiling grep-cli v0.1.6
Compiling textwrap v0.11.0
Compiling console v0.15.0
Compiling dirs-next v2.0.0
Compiling cc v1.0.72
Compiling clap v2.34.0
Compiling libz-sys v1.1.3
Compiling onig_sys v69.7.1
Compiling sys-info v0.9.1
Compiling libgit2-sys v0.12.26+1.3.0
Compiling bugreport v0.4.1
Compiling serde_derive v1.0.136
Compiling git-version-macro v0.3.5
Compiling thiserror-impl v1.0.30
Compiling onig v6.3.1
Compiling git2 v0.13.25
Compiling bat v0.20.0 (/tmp/portage/sys-apps/bat-0.20.0/work/bat-0.20.0)
Compiling git-version v0.3.5
Compiling thiserror v1.0.30
Compiling serde v1.0.136
Compiling bincode v1.3.3
Compiling plist v1.3.1
Compiling serde_json v1.0.74
Compiling serde_yaml v0.8.23
Compiling clircle v0.3.0
Compiling syntect v4.6.0
Finished release [optimized] target(s) in 2m 08s
Installing /tmp/portage/sys-apps/bat-0.20.0/image/usr/bin/bat
Installed package `bat v0.20.0 (/tmp/portage/sys-apps/bat-0.20.0/work/bat-0.20.0)` (executable `bat`)
warning: be sure to add `/tmp/portage/sys-apps/bat-0.20.0/image/usr/bin` to your PATH to be able to run the installed binaries
>>> Completed installing sys-apps/bat-0.20.0 into /tmp/portage/sys-apps/bat-0.20.0/image
* Final size of build directory: 521044 KiB (508.8 MiB)
* Final size of installed tree: 4240 KiB ( 4.1 MiB)
Note that only a few crates is recompiled.
Version
> $HOME/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/bin/cargo version --verbose
cargo 1.61.0-nightly (109bfbd05 2022-03-17)
release: 1.61.0-nightly
commit-hash: 109bfbd055325ef87a6e7f63d67da7e838f8300b
commit-date: 2022-03-17
host: x86_64-unknown-linux-musl
libgit2: 1.4.2 (sys:0.14.2 vendored)
libcurl: 7.80.0-DEV (sys:0.4.51+curl-7.80.0 vendored ssl:OpenSSL/1.1.1m)
os: Linux 2.8 [64-bit]
> /usr/bin/cargo version --verbose
cargo 1.59.0-nightly
release: 1.59.0-nightly
host: x86_64-unknown-linux-musl
libgit2: 1.3.0 (sys:0.13.23 vendored)
libcurl: 7.82.0 (sys:0.4.51+curl-7.80.0 system ssl:OpenSSL/1.1.1n)
os: Linux 2.8 [64-bit]
The fingerprint that determines a rebuild is affected by several things. I guess at least the version info of invoked rustc changed when switching to a super user, due to PATH changed. That behaviour is expected because when across different version of rustc, we need to recompile.
The Cargo Book contains a whole paragraph about knowing the reason why a rebuild occurs. It's not comprehensive but usable for now. You could test it to help verify my assumption.
If I do want to avoid recompilation when sudo cargo, I might set CARGO_HOME=/path/to/my/cargo/bin/dir and then invoke cargo in order to control which rustc to be called. (I haven't tried it, though)
I find that the exa case is not a fingerprint bug, but (maybe) a old bug of feature resolver v1.
I hack cargo to print build plan when run cargo install subcommand, and compare this output with build plan from cargo build -Z unstable-options --build-plan
The different is, in build plan of cargo install, --cfg feature="extra_traits" is enabled for libc and --cfg feature="full" is enabled for syn, both of them is pulled by a dev-dependency ( nix need extra_traits feature of libc, serial_test_derive need full feature of syn)
So the rebuild is caused by behavior mismatch between cargo install and cargo build in feature resolver v1.
Hmm… interesting. Haven't tried out your new discovery, but I can reproduce the steps in the PR description and the example has no dependency, so these two might be different issues.
If you can set identical CARGO_HOME or PATH for the superuser account, the build should be fresh and won't trigger any re-build I guess (for the example in PR description).
It looks like this is using cargo install without the --locked option. cargo install will by default ignore Cargo.lock, which is probably what you are seeing here.
It's cause by this:
https://github.com/rust-lang/cargo/blob/403c6bd257e4501ec59d51aadb347d83ca0f2fb3/src/cargo/ops/cargo_install.rs#L755
And introduced in https://github.com/rust-lang/cargo/commit/d1ef031a6272e9eee55d97189c1f02bd24f1504a
For someone who encountered the same problem, here's a quick solution.
sudo env "CARGO_HOME=$HOME/.cargo" cargo build # or run or install
