actions
Configurable save cache on failure
Currently this cache action will only save caches if all tests succeed. In many cases this is desirable behavior. However I have some projects with long build times and flakey test suites. It would be very helpful if I could configure the cache to be saved regardless of the test suite success or failure.
I have created a fork of this project action to set the post-if to always().
Is it possible to make the cache policy configurable? Or to pass post-if as an argument from the cache configuration?
I also desperately need this. For my use cases (grading of student assignments), it is expected that at least some of the tests will fail most of the time, so having a completely successful run is rare. This means caching is basically useless, as the cache will never actually be persisted.
@gerbal I've tried your fork, but it also doesn't seem to work. The post-action simply never runs (it stays a grey square in the log UI), same as with the original cache action.
Bump. I have some caches that take a long time to rebuild, and sometimes they may need to be rebuilt due to external events right in the middle of other changes that may cause subsequent steps to fail. So it's a big waste of time to have to then disable and reenable all of the following steps to be able to save the cache. A totally configurable 'pass-if' option would be a great solution, but if for some reason that's difficult to implement, maybe we could have a 'save-after' option where you could pass the id of the step that builds the thing that needs to be cached ? Then instead of running the post cache step after all other steps, it could run right after the 'save-after' step, and then that would make it succeed regardless of any subsequent steps that might fail ?
Next fork with latest upstream changes (including the new 5GB cache limit): https://github.com/marketplace/actions/always-upload-cache
For anyone interested, you are welcome to try out my advanced cache action built on top of this repo. Simply specify your cache targets in a config file, and finishes restoration, build, and save as many caches as you want in one step:
steps:
- uses: actions/checkout@v2
- uses: ktmud/cached-dependencies@v1
with:
run: |
cache-restore npm
npm install
cache-save npm
cache-restore pip
pip install -r requirements.txt
cache-save pip
https://github.com/ktmud/cached-dependencies#speficy-when-to-restore-and-save
Seems like it would make a lot of sense in some circumstances.
Say I have following steps:
- install dependencies (success)
- check code style (fail)
- post cache (skip)
It would make sense if post cache could happen conditionally on success of dependencies install, code style step is irrelevant to it.
This is very much needed. Even another action which only uploads would be fine.
Something like:
- name: Upload cache because failure
if: failed()
uses: actions/upload-cache
with:
path: dependencies/
key: 123
+1 yes please! Seems like a fairly minor addition that would make a bit difference
+1 I agree, having this as an option to cache even on a failure. Would be nice, because in docker build workflow this woud allow to get the working layers out of the cache and reduce much build time if the build fails somewhere.
@sidey79 @ben-spiller @ollydev @glau2 Is it possible to add post-if to the CI? Like this: post-if: always().
I'm not sure if that is available to users of the action or only available to the developer in action.yml.
@Rarst yes, that's what I suspect. post-if is not documented in the user action guide as part of the spec.
You could fork it yourself and change it to always().
FYI: such fork already exists: https://github.com/pat-s/always-upload-cache
You can also reference env vars in the post-if condition:
post-if: success() || env.ALWAYS_SAVE_CACHE == 'true'
(I would love to replace the env var with an input here, but that wasn't supported when I last tested.)
@dhadka Inputs and env vars are basically the same thing:
From: https://raw.githubusercontent.com/actions/cache/main/dist/save/index.js
/**
* Gets the value of an input. The value is also trimmed.
*
* @param name name of the input to get
* @param options optional. See InputOptions.
* @returns string
*/
function getInput(name, options) {
const val = process.env[`INPUT_${name.replace(/ /g, '_').toUpperCase()}`] || '';
if (options && options.required && !val) {
throw new Error(`Input required and not supplied: ${name}`);
}
return val.trim();
}
exports.getInput = getInput;
Why would you rather use an input instead of an environment variable? The input would be static but an environment variable you can change whenever you want.
@eyal0 👍 true. This is getting off topic 😄 but I prefer inputs because:
-
This keeps all the inputs to the action defined in one spot instead of split across inputs and env vars, especially when it changes the functionality of the action.
-
If you need or want to use an env var, you can always pass the env var to the input:
with: always-save: ${{ env.ALWAYS_SAVE_CACHE }} -
Probably most importantly, inputs are well-defined for an action. You can provide a description, make inputs required or optional, and set default values in
action.yml. If you had a typo, for example, an input would show a warning or error whereas an env var would silently ignore the typo. This also lets us auto-generate documentation for each action when using the workflow editor.
But for this issue since we can't use inputs in the post-if condition, this is a moot point.
Yeah. If you want, you could use my fork and just change the "success()" to "always()".
https://github.com/actions/cache/pull/498#issuecomment-753804797
It lets you control the behavior of the post action with an environment variable. So you could "always" run the post action and then have an environment variable that is set to true or false to control whether or not to update the cache. There are likely dozens of solutions similar to mine.
The authors of actions/cache ought to look at what the common forks of this repo are trying to do and incorporate the features.
@pat-s please add a v2 tag to your project, matched to the latest v2* release.
This will allow drop-in compatibility with actions/cache@v2
I made a fork of this repository that gives you full control over when the cache is saved. This allows you to both set your own if: always() (or success or failure or what you want) as well as run the save action where/when you want. Docs here: https://github.com/MartijnHols/actions-cache
For the OP it sounds like you run install and tests in a single job. If you place the save action after the step that generates the things you want to cache, you can achieve this without always. For example:
name: Build app
on: push
jobs:
install:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Restore "node_modules" from cache
id: cache
uses: martijnhols/actions-cache/restore@v3
with:
path: node_modules
key: ${{ runner.os }}-node_modules-${{ hashFiles('yarn.lock', 'patches') }}
restore-keys: ${{ runner.os }}-node_modules
- name: Install dependencies
if: steps.cache.outputs.cache-hit != 'true'
run: yarn install
- name: Save "node_modules" to cache
if: steps.cache.outputs.cache-hit != 'true'
uses: martijnhols/actions-cache/save@v3
with:
path: node_modules
key: ${{ steps.cache.outputs.primary-key }}
- name: Run flaky tests
run: yarn test
If you want your cache to be saved regardless of a previous failure you can change the if: steps.cache.outputs.cache-hit != 'true' line into if: always() && steps.cache.outputs.cache-hit != 'true'.
An alternative workaround, which does not require maintaining a forked version of actions/cache@v3, is to check out the action repo manually, modify action.yml, and run the modified action from its local path:
- name: Checkout actions/cache@v3
uses: actions/checkout@v3
with:
repository: actions/cache
ref: v3
path: .tmp/actions/cache
- name: Make actions/cache@v3 run always, not only when job succeeds
# Tweak `action.yml` of `actions/cache@v3` to remove its `post-if`
# condition, making it default to `post-if: always()`.
run: |
sed -i -e '/ post-if: /d' .tmp/actions/cache/action.yml
- name: Cache data
id: cache
uses: ./.tmp/actions/cache
with:
...
You can see a minimal example of a full workflow in https://github.com/mxxk/gh-actions-cache-always. The modified actions/cache@v3 command still saved the cache even though the job https://github.com/mxxk/gh-actions-cache-always/runs/5847143256 failed.
(Of course, this is only a hack, since the sed command which finds and deletes the post-if: line from action.yml is quite janky.)
I've just updated my fork to v3.0.1: https://github.com/marketplace/actions/always-upload-cache
Any progress on this one? I have the same problem as mentioned in https://github.com/actions/cache/issues/92#issuecomment-562204290.
Would this feature be officially supported?
+1
We're running a test suite with Hypothesis, a property based testing tool. The database should be saved (cached) on failed runs, so failed tests are replayed on subsequent runs.
This would be extremely useful for my team. We use Next.js. The process goes something like this.
- Restore the Next build cache
- Run the Next build (hopefully with cache)
- Run Cypress tests (which can be flakey)
- Save the Next build cache.
It would be great to save the Next cache even if the Cypress tests fail. There's a high likelihood that the Cypress tests fail due to flakiness, but sometimes we just need to update the Cypress tests. That change doesn't invalidate the build cache, so it would be nice to have that speedier build on the second run.
We have the same use case @trent-boyd for this: failing / flaky end-to-end tests are really hard to debug due to having to re-run build processes: by allowing to cache builds even on failure would speed up this considerably.
I made https://github.com/mxxk/gh-actions-cache-always into a reusable action, which patches the original actions/cache to change the post-if: 'success()' predicate to post-if: 'success() || failure()'.
.github/actions/cache-always/action.yml:
Show file contents
name: 'Cache Always'
description: 'Cache artifacts like dependencies and build outputs to improve workflow execution time'
inputs:
path:
description: 'A list of files, directories, and wildcard patterns to cache and restore'
required: true
key:
description: 'An explicit key for restoring and saving the cache'
required: true
restore-keys:
description: 'An ordered list of keys to use for restoring stale cache if no cache hit occurred for key. Note `cache-hit` returns false in this case.'
required: false
upload-chunk-size:
description: 'The chunk size used to split up large files during upload, in bytes'
required: false
outputs:
cache-hit:
description: 'A boolean value to indicate an exact match was found for the primary key'
value: ${{ steps.cache.outputs.cache-hit }}
runs:
using: 'composite'
steps:
# Instead of running `actions/cache@v3` directly, check it out locally.
- name: Checkout actions/cache@v3
uses: actions/checkout@v3
with:
repository: actions/cache
ref: v3
path: ./.github/.tmp/cache-always/actions/cache
- name: Patch actions/cache@v3 to make it cache data also when the job fails
run: |
sed -i -e 's/post-if:.*$/post-if: "success() || failure()"/' ./.github/.tmp/cache-always/actions/cache/action.yml
shell: bash
- name: Cache
id: cache
uses: ./.github/.tmp/cache-always/actions/cache
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}
restore-keys: ${{ inputs.restore-keys }}
upload-chunk-size: ${{ inputs.upload-chunk-size }}
You can use it by saving the above action into its recommended location and writing:
uses: ./.github/actions/cache-always
instead of
uses: actions/cache@v3
e.g.
- name: Setup pnpm cache
uses: ./.github/actions/cache-always
with:
path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
Edits: 2022-11-11: run on success() || failure(), to prevent caching on cancelled()
+1, it seems like there is a pretty clear desire for this. It would be really great to get this as a configurable option in the 'official' actions/cache.
