Fails to handle environment variables changes from build.rs

[Guiguiprim]

Issue

When defining environment variables in a build script within a workspace, they are not correctly handle by the cache. This is visible in non incremental build (in release with default config).

Setup:

• Have a build script defining an environment variable from an outside input (a file, with rerun on it) and use it in a crate
• Use that crate in a binary crate
• Build and run in release
• Edit the outside input
• Build and run in release (the build script is triggered due to the rerun)
• => The execution should use the new value but did not: cache issue, environment variable changes not seen

Minimal reproducer

# extract archive, cd into extracted folder
> RUSTC_WRAPPER=cachepot cargo r --release
# should output: "Value is: something else"

# Edit the parameter read by the build script
> echo "anything else" > parameter.txt

> RUSTC_WRAPPER=cachepot cargo r --release
# should output: "Value is: anything else" but output the same as last time

cachepot-env-vars.tar.gz

[montekki]

This is an interesting case, the solution would be whenever something like cargo:rerun-if-changed and its relatives is encountered, the referenced data has to be counted into the hash.

Right now cargo sees that the file in question has changed and issues a build command to cachepot. cachepot sees that the hash value exists in the cache and hands it back.

The problem is how to get this info outside of cargo into cachepot since AFAIU it is strictly inside-cargo business and it does not communicate this info anywhere else. I will look into it maybe the best way is just to launch the built build.rs from inside cachepot and parse the output.

[Guiguiprim]

I discovered this issue with the following use case: including the current commit-id in a --version output. But it could impact other things.

I was wondering if including the build.rs output file into the hash (as it contains all the defined env-vars) would do the tricks.

[Guiguiprim]

I will look into it maybe the best way is just to launch the built build.rs from inside cachepot and parse the output.

If that helps, the output of build.rs is saved into a file name output somewhere inside the target folder. (ex: target/x86_64-pc-windows-msvc/release/build/my_lib/output).

Actually, I don't think we need to care about rerun-if-changed. The build script does re-run, so only caring about output changing should be enough (and possibly cover other things in the same spirit).

[drahnr]

Does your build script create any files or update any file's content?

[Guiguiprim]

Does your build script create any files or update any file's content?

I do not create any file. But as previously stated, cargo does save its output into a file.

[drahnr]

@montekki: If the path is predictable, we could simply add this to the cached paths if the manifest contains a build.rs target/x86_64-pc-windows-msvc/release/build/my_lib/output for files to be tracked.

[Guiguiprim]

If the path is predictable,

I simplified a bit, its more like target/x86_64-pc-windows-msvc/release/build/my_lib-c89d522335dccf66/output.

I guess that -c89d522335dccf66 can be deduced somehow.

[montekki]

@drahnr I think has to be done:

../target/debug/build/my_lib-21274d7f2c1f2efb/build-script-build
cargo:rerun-if-changed=../parameter.txt
cargo:rustc-env=MY_PARAMETER=anything else

And parse all the necessary outputs. the build-script-build is a pre-defined hardlink to the built build.rs. The vars and their meanings are defined here

[daladim]

Hello. Is there any update on this issue? Otherwise, I may try to fix this in the days to come

[daladim]

Hmmm, well I'm not sure I'll have enough free time to see this after all. I'll have a try but maybe I won't be able to complete this fix :-(

If that helps, I'm sharing the workaround that currently works (at least for my own needs): all the environment-related behaviour of my build.rs is to write some files, that then get include!d in my source files. Indeed (and unlinke env vars), the files used by include!, include_str! and inlcude_bytes!, are correctly tracked by cachepot.

[drahnr]

There were some slowdowns related to the invasion of Russia into the Ukraine. If you can create a PR, even if it's just a rough draft, would be helpful.