Caching on CI does not seem to work

[curiousleo]

Both the nix build and the cargo build appear to always build all dependencies from scratch, which leads to build times between 10 and 30 minutes.

Example: https://travis-ci.org/target/lorri/builds/626109465

[Profpatsch]

$ nix-build -A allBuildInputs shell.nix > ./shell-inputs
unpacking 'https://github.com/NixOS/nixpkgs/archive/3140fa89c51233397f496f49014f6b23216667c2.tar.gz'...
these derivations will be built:
  /nix/store/kk9c7kkgy344dkfwrbbdizlbi2y2icxh-clippy-1.39.0.drv
these paths will be fetched (447.85 MiB download, 1211.02 MiB unpacked):
  /nix/store/0vcqyg4cadpns1l935h18ram5ngqqsgq-aws-sdk-cpp-1.7.90
  /nix/store/0zgfzj164xhhc56bfgk9q53aqm5hgk9n-patch-2.7.6

It’s building clippy on every run, which takes a long time. Maybe that was introduced with the last rust update?

[curiousleo]

Maybe that was introduced with the last rust update?

That's definitely possible. I can see how changing the clippy dependency would lead to clippy being built once (or once per platform) on Travis. But why keep rebuilding it on every CI run? It looks to me like the "cache" build step is meant to cache cargo outputs: https://github.com/target/lorri/blob/master/.travis.yml.nix#L103 - but somehow clippy and other dependencies are being built every time anyway.

[curiousleo]

For nix-based builds, part of the puzzle is of course that we've disabled cachix on macOS due to this issue: https://github.com/cachix/cachix/issues/228#issuecomment-533634704.

Why is rustPackages.clippy only available in source form, but not in compiled form, from cache.nixos.org?

Edit: see https://github.com/NixOS/nixpkgs/pull/77320, https://github.com/NixOS/nixpkgs/pull/77327, https://github.com/NixOS/nixpkgs/issues/77358.

[Profpatsch]

For nix-based builds, part of the puzzle is of course that we've disabled cachix on macOS due to this issue: cachix/cachix#228 (comment).

That’s fixed since #340

Build times seem to be down to ~8 minutes, which is not rosy, but at least the nix dependencies are cached again.

[Profpatsch]

The caching for the macos nix build still has some problems: https://travis-ci.org/target/lorri/jobs/657713960?utm_medium=notification&utm_source=github_status