cache
cache copied to clipboard
actions
Feature request: option to update cache
Problem Description Currently, the cache action either restores an existing cache on cache-hit, or generates a missing cache on cache-miss. It does not update the cache on cache-hit.
This works well for caching static dependencies, but not for caching build artefacts.
Proposed Solution
Add an option allowing the user to enable cache updates.
This should be false by default to retain backwards-compatibility.
uses: actions/cache@v2
with:
path: ccache
key: ${{ matrix.CONFIG }}-${{ matrix.CXX }}-${{ matrix.TYPE }}
update: true # <~~ explicitly request an update
Motivation Some programming languages benefit greatly from build caching. C++ in conjunction with ccache is the prime example. Using caching commonly decreases compilation times by at least 70%. Medium-sized projects easily take 20 minutes to compile. ccache also manages the cache size itself and automatically removes obsolete entries, thus the cache won't explode with continuous updates.
It also saves time and money for both user and provider. The environment will be happy too.
I'm having this issue when trying to use @actions/cache for reducing update time of the built-in MSYS2 installation on windows-latest virtual environments. The virtual environments are outdated quite fast, and currently 5503.95 MiB need to be downloaded and installed on each job. It takes 8-10 min.
As commented in eine/setup-msys2#23, I'm trying to save /var/cache/pacman/pkg/. However, actions/checkout@v2 allows to do it once only. Later executions skip it: https://github.com/eine/setup-msys2/runs/737062486?check_suite_focus=true#step:12:2
Post job cleanup.
Cache hit occurred on the primary key msys, not saving cache.
Then, I tried using the npm package: https://www.npmjs.com/package/@actions/cache. Unfortunately, it fails: https://github.com/eine/setup-msys2/runs/738298072?check_suite_focus=true#step:4:116
##[error]reserveCache failed: Cache already exists. Scope: refs/heads/tool-cache, Key: msys2-nokey, Version: 000d31344dacf74d63d9e122f85409f68c5697c2aa32c5626452e8301c5d0c66
As an alternative to updating an existing key, it would be feasible to remove it explicitly (ref #340).
This would also be super handy for persisting the test cache when using GitHub Actions in go projects. For example, I could download the latest test cache, run my tests then update the existing cache with the results of the tests in the current run.
This way, I can have a global test cache across all my workflow runs.
This also applies to things like webpack loader cache, small, have their own key management, needs to be updated each time
This would be super valuable for monorepo's where each subproject has its own dependencies it wants to cache and build so that upstream projects have faster build times.
I ran into the expectation that this was already the default behavior - so I wrote #392 under that assumption
Is there any workaround for forcing the update of the cache even on a hit? I can't think of any way... In my case it would reduce the compilation time from 20 minutes to 3 minutes if I could use ccache for QT/C++.
Maybe save as ccache-${{ github.run_id }} and restore with restore key ccache-.
github.run_id is unique id for the workflow run, so every time a new cache is saved.
When restoring you will never have an exact match but then the ccache- restore key will restore the latest one that started with that string and in the end create a new one with the current state.
@Vampire that's brilliant, thanks mate! You are right, forgot about the pattern matching that the cache finding does. Perhaps this is then a non-issue and your solution is the intended way of doing things?
Nah, that's merely a work-around. It will fill up your 5 GiB of cache and then evict things that might not have been evicted if the cache would have been updatable.
@HebaruSan It won't save anything at all.
https://github.com/actions/toolkit/blob/73d5917a6b5ea646ac3173cfceb727ee914ff6ed/packages/cache/src/cache.ts#L166-L175
This is absolutely required, I use it to speed up eslint and my eslintcache file always gets updated on run even if it restores. I need to force save this file somehow after the run is complete.
Also helps Typescript tsc --build --incremental (on by default with Project References), caching the outDir and any *.tsbuildinfo files. We don't need to key these based on any src files, need the results from a previous build. tsc will handle changes in src (well, for the most part https://github.com/microsoft/TypeScript/issues/16057).
@HebaruSan It won't save anything at all.
https://github.com/actions/toolkit/blob/73d5917a6b5ea646ac3173cfceb727ee914ff6ed/packages/cache/src/cache.ts#L166-L175
It's now have been updated and new cache will override old ones.
Caching dependencies to speed up workflows
GitHub will remove any cache entries that have not been accessed in over 7 days. There is no limit on the number of caches you can store, but the total size of all caches in a repository is limited to 10 GB. If you exceed this limit, GitHub will save your cache but will begin evicting caches until the total size is less than 10 GB.
@PathogenDavid @HebaruSan