Advice on toolchain caching

[HadrienG2]

So, I have a workflow with a relatively big build matrix composed of many short jobs. As a result, toolchain downloads are one of the CI perf bottlenecks, which leads me to wonder if I couldn't speed things up by caching the toolchain.

Unfortunately, all my attempts at doing so failed so far, IIUC due to a mixture of file permission issues and my lack of knowledge on the directory layout used by the "toolchain" action. I think it would help if you either explained how to do it in the toolchain action's documentation, or clarified that it cannot be done in the current rustup installation directory layout if that's the issue.

[djc]

It seems we could use this: https://github.com/actions/cache/blob/master/examples.md#rust---cargo

[HadrienG2]

I tried to use it in various ways, but did not quite get it to work. Unfortunately, I did not take notes at the time, and the GH Actions environment is not amenable to quick replication, but from memory...

• I tried variants of the usual ~/.rustup directory and got errors saying that it didn't exist.
• I noticed that the toolchain output mentioned subdirectories of /usr/share/rust and tried caching things like /usr/share/rust/.rustup, but all I got was lots of file permission errors, most likely because I'm not allowed to write in there.

Since the GH action environment seems to be peculiarly configured in various ways, and does not provide tools for easy experimentation like similar docker containers that you can run on your machine or interactive SSH sessions, documentation and advice on what can be cached and where it's located would be nice.

[jhpratt]

For what it's worth, I just tried using action/cache@v1 in combination with overriding permissions (to avoid errors), and found minimal difference in speed. It seems the network is the restricting factor, not anything else.

[lcdr]

So, for some reason the ~/.rustup directory itself won't be cached properly, but caching its contents works. The following setup works for me:

- name: Cache toolchain
  uses: actions/cache@v2
  with:
    path: |
      ~/.rustup/toolchains
      ~/.rustup/update-hashes
      ~/.rustup/settings.toml
    key: toolchain-${{ hashFiles('rust-toolchain') }}

This caches based on the contents of the rust-toolchain file.

[HadrienG2]

I suspect that ~/.rustup might be a symlink, which would explain why caching it doesn't work while caching its content does. But indeed, I ended up using a similar strategy as yours, and with cache v2 this leads to a net speed improvement at last!

      - name: Cache toolchain
        if: github.event_name != 'schedule'
        uses: actions/cache@v2
        with:
          path: |
            ~/.rustup/settings.toml
            ~/.rustup/toolchains/${{ matrix.rust }}-*
            ~/.rustup/update-hashes/${{ matrix.rust }}-*
          key: run-toolchain-rust_${{ matrix.rust }}

[remagpie]

Running ls -alh on the home folder shows the following result:

Run ls -alh ~/
total 60K
drwxrwxrwx  11 runner docker 4.0K Apr 19 12:10 .
drwxr-xr-x+  5 root   root   4.0K Apr 16 17:25 ..
-rwxrwxrwx   1 runner docker  220 Apr 16 17:25 .bash_logout
-rwxrwxrwx   1 runner docker   67 Apr 16 17:25 .bash_profile
-rwxrwxrwx   1 runner docker 3.7K Apr 16 17:25 .bashrc
lrwxrwxrwx   1 runner docker   22 Apr 16 17:25 .cargo -> /usr/share/rust/.cargo
drwxrwxrwx   2 runner docker 4.0K Apr 16 17:25 .composer
drwxrwxrwx   4 runner docker 4.0K Apr 19 12:10 .config
drwx------   2 runner docker 4.0K Apr 16 17:28 .docker
lrwxrwxrwx   1 runner docker   17 Apr 16 17:25 .ghcup -> /usr/local/.ghcup
drwxrwxrwx   6 runner docker 4.0K Apr 16 17:25 .nvm
-rwxrwxrwx   1 runner docker  807 Apr 16 17:25 .profile
lrwxrwxrwx   1 runner docker   23 Apr 16 17:25 .rustup -> /usr/share/rust/.rustup
drwxr-xr-x   2 runner root   4.0K Apr 16 17:28 factory
drwxr-xr-x   2 runner docker 4.0K Apr 19 12:10 perflog
drwxr-xr-x   4 runner root   4.0K Apr 16 17:28 runners
drwxr-xr-x   2 runner root   4.0K Apr 16 17:28 warmup
drwxr-xr-x   6 runner root   4.0K Apr 19 12:10 work

Caching /usr/share/rust/.rustup(the destination folder of symlink at ~/.rustup) worked for me.

[ebkalderon]

Turns out that caching the ~/.rustup directory itself now works perfectly fine on actions/cache@v3, so there's no need to drill down into its contents anymore, as the above 3 comments all suggest. Caching the toolchain is pretty simple these days:

- id: cache-rustup
  name: Cache Rust toolchain
  uses: actions/cache@v3
  with:
    path: ~/.rustup
    key: toolchain-${{ matrix.os }}-${{ matrix.rust-version }}

- if: ${{ steps.cache-rustup.outputs.cache-hit != 'true' }}
  name: Install Rust toolchain
  uses: actions-rs/toolchain@v1
  with:
    profile: minimal
    toolchain: ${{ matrix.rust-version }}
    override: true
    components: clippy, rustfmt

Given a list of runner labels (matrix.os) and a list of valid Rust toolchain versions (matrix.rust-version), the above code should create an OS- and version-specific toolchain cache, e.g. toolchain-ubuntu-latest-1.67.1.

Technically, the matrix.os component above can be safely omitted, because caches cannot be reused between different runners (source).