GitLab cache problem

[kuviman]

So, I've been using GitLab CI to automatically deploy native versions as well as WebAssembly versions of my application. The problem is, that if I enable cache to speed up compilation, cargo-web produces outdated binary. Native builds with cargo build work fine.

I've managed to get a minified example here: https://gitlab.com/kuviman/cargo-web-cache-test

This is just a simple hello-world app:

fn main() {
    println!("This is my test. My test is amazing.");
}

Here's what I do in the script:

cargo web deploy --release --target wasm32-unknown-emscripten

And I cache the entire target directory:

cache:
  paths:
    - target

The first build (when nothing is yet cached) goes fine, but if I change the text in the example, I see that cargo web still produces old wasm file (I check it using node)

This is the job's (https://gitlab.com/kuviman/cargo-web-cache-test/-/jobs/114564134) log:

$ cargo web build --release --target wasm32-unknown-emscripten
info: downloading component 'rust-std' for 'wasm32-unknown-emscripten'
info: installing component 'rust-std' for 'wasm32-unknown-emscripten'
   Compiling cargo-web-cache-test v0.1.0 (/builds/kuviman/cargo-web-cache-test)
    Finished release [optimized] target(s) in 1.29s
$ cargo web deploy --release --target wasm32-unknown-emscripten
    Finished release [optimized] target(s) in 0.00s
The `cargo-web-cache-test` was deployed to "/builds/kuviman/cargo-web-cache-test/target/deploy"!
$ cat src/main.rs
fn main() {
    println!("This text is in code but not in wasm binary");
}
$ (cd target/deploy && node cargo-web-cache-test.js)
This is my test. My test is amazing.
$ (cd target/wasm32-unknown-emscripten/release && node cargo-web-cache-test.js)
This is my test. My test is amazing.
$ cargo run
   Compiling cargo-web-cache-test v0.1.0 (/builds/kuviman/cargo-web-cache-test)
    Finished dev [unoptimized + debuginfo] target(s) in 0.94s
     Running `target/debug/cargo-web-cache-test`
This text is in code but not in wasm binary

I am pretty sure it's an issue with cargo-web, and I wonder if this also can be reproduced on Travis or is it GitLab-specific

[koute]

Hmm... this is interesting. Although if it reproduces on wasm32-unknown-emscripten it may simply be a rustc bug. (I don't do too much acrobatics on wasm32-unknown-emscripten, unlike on wasm32-unknown-unknown.) I'm guessing the issue here might be that the process of restoring the cache changes the mtime of some of the files in relation to others which makes the build system think that there is nothing to rebuild.

Can you do a retest and run this command before cargo web build and after? (Both with a clean cache and a full cache?)

find . -type f -print0 | xargs -0 ls -l --time-style="+%F %T.%N"

[kuviman]

Ok, and I've also changed the message to be generated so we see that wasm file contains previous build easier:

$ date "+%F %T.%N" > message
...
$ cat message
2018-10-30 23:16:39.281211204
$ (cd target/wasm32-unknown-emscripten/release && node cargo-web-cache-test.js)
2018-10-30 23:12:23.156200724

Job with cache cleared: https://gitlab.com/kuviman/cargo-web-cache-test/-/jobs/114620889 Job with cached produced by previous one: https://gitlab.com/kuviman/cargo-web-cache-test/-/jobs/114621656

I see that times of files in target/ look like real ones of the previous build, and source files are just cloned, so source files are newer than built app, and everything should be rebuilt correctly.

I am pretty sure that actual compilation is being done. In my real project that is a repo consisting of several crates, cargo shows that all these crates are being compiled.

I assume that multiple files are generated when compiling (crate-$hash directories with different hashes), and when linking wrong files are being used.

Although I see no problem when using it on my PC.

This may be a rustc bug indeed. I should test this without cargo-web I suppose. wasm32-unknown-unknown does work, it looks like, but I prefer using stable.

[kuviman]

Sorry, forgot to include find output after build. Here:

1. https://gitlab.com/kuviman/cargo-web-cache-test/-/jobs/114627055
2. https://gitlab.com/kuviman/cargo-web-cache-test/-/jobs/114627630

[kuviman]

It is also strange that with clean cache, cargo is building the app twice: first taking 1.5 sec (little time), and second taking 1.5 min (long time).

With full cache, there is only one build, and only the short (1.5 sec) one

Isn't it true that most time here should be spend in emscripten, so this should indicate that emcc is not run with cache? Double build is coming from cargo-web, right?

[kuviman]

Hello! I was trying to see if this would work with rustc, and it does work properly.

But there is this strange thing: cargo-web says (this job, clean build no cache involved):

cargo_web::cargo_shim: No artifacts were generated yet build succeeded; retrying...

And in a log of build without cargo-web you can see that there are artifacts, although I tried to replicate the environment exactly, so the only difference should be in how emscripten is set up.

According to this commit there is some issue with newly installed emscripten, although I can't find more info on that.

So, what happenes is I think this: this bug (bug in Emscripten?) is still happening, but rustc thinks the old wasm is still the output. And even double build does not help. What helps though is removing .js & .wasm files from the target/../../deps directory

[koute]

Yes, there is a bug, either in rustc or in Emscripten where immediately after a clean installation of Emscripten the first build produces no artifacts. I have no idea why.

In your job without cargo-web are you sure the docker image with the Emscripten doesn't have the Emscripted already "warmed up"? E.g. ~/.emscripten, ~/.emscripten_cache, ~/.emscripten_cache.lock, ~/.emscripten_sanity should not exist.

[remram44]

I think I'm running into this problem. I am building locally (not CI) but via a Docker image, and the artifact is not updated

docker run -ti --rm -v $(pwd):/src -w /src remram/rust-cargo-web:latest cargo web deploy --release

I am also seeing two builds:

...
   Compiling piston v0.49
   Compiling piston2d-opengl_graphics v0.69.0
   Compiling client-piston v0.1.0 (/src)
    Finished release [optimized] target(s) in 44.13s
   Compiling client-piston v0.1.0 (/src)
    Finished release [optimized] target(s) in 47.85s