cache icon indicating copy to clipboard operation
cache copied to clipboard

Skip post cache

Open nroose opened this issue 4 years ago • 9 comments

We have several jobs that we run at once, and they use the same cached files, and if they update them, they would make the same updates. We don't need to update the cache for all of them, just for one. And actually some of them usually fail because they are all trying to update the cache at the same time. So we would like to skip the cache for all the other jobs.

We may also only want to update the cache for specific branch patterns to save time on branches that won't likely be in a state that gets a valid cache hit anyway, so they shouldn't save the cached files.

Other than that, it works great!

Thanks!

nroose avatar Nov 14 '20 06:11 nroose

Would be good to have this as well as the option to force a cache update (#342)

rcowsill avatar Dec 05 '20 23:12 rcowsill

Here's my solution: https://github.com/actions/cache/pull/498#issuecomment-753804797

eyal0 avatar Jan 04 '21 07:01 eyal0

Another use case: I have a number of jobs sharing a cache (node_modules etc.). In one of them, I want to trim these files (to build smaller Docker images), but if I delete parts of the cached files, they will be uploaded, which means the cache is wrong for all other jobs.

I want to treat the cache as pull-only and discard local changes—rather like the "pull" behaviour you can do with Gitlab CI.

haggholm avatar Dec 22 '21 23:12 haggholm

Would be good to have this as well as the option to force a cache update

There is an open issue in the future roadmap that will allow cache deletion. With cache deletion, it should be possible to delete a cache that is no longer required so that in the subsequent steps, it can be recreated again.

aparna-ravindra avatar May 16 '22 12:05 aparna-ravindra

I want to treat the cache as pull-only and discard local changes

If there is a cache hit on the primary key, the post action does not save the cache. Therefore, even if you make local changes, it should not affect the already cached content. Effectively, the local changes are discarded once the job finishes.

aparna-ravindra avatar May 16 '22 12:05 aparna-ravindra

And actually some of them usually fail because they are all trying to update the cache at the same time

This is expected because of the jobs running in parallel. However, only one job will succeed in creating the cache while others will simply exit.

aparna-ravindra avatar May 16 '22 12:05 aparna-ravindra

I want to treat the cache as pull-only and discard local changes

If there is a cache hit on the primary key, the post action does not save the cache. Therefore, even if you make local changes, it should not affect the already cached content. Effectively, the local changes are discarded once the job finishes.

It's a shame. The easiest paradigm would be to allow users to treat the cache like a regular file system. The users just specify the filename and if they want to read it, write it, overwrite it, etc. And that's it.

Currently it's readable and you can write it but you can only write it if it doesn't already exist. No overwrite is allowed.

I know that this would take a change not just in the action but also in the server because it's actually the server that is not allowing overwrite.

eyal0 avatar May 16 '22 14:05 eyal0

Currently it's readable and you can write it but you can only write it if it doesn't already exist. No overwrite is allowed.

This is correct. This open issue in the roadmap will allow cache deletion. So, you will be able to delete+create new to update an existing cache.

aparna-ravindra avatar May 17 '22 02:05 aparna-ravindra

Hi 👋🏽 If there are no more open questions here, can we close the issue?

aparna-ravindra avatar Jun 07 '22 09:06 aparna-ravindra

We're looking for this functionality -- we have a large cache which we only want to update on the main branch. If a whole bunch of branches update it, we'll quickly start evicting useful caches due to the 10Gb limit.

It's unlikely it'll be an exact match on the branch (think Dependabot package updates) so the current design will always create a new cache, which we don't want. But we do want the benefit of the cache being a 95% match to what we need.

simonrussell avatar Dec 02 '22 00:12 simonrussell

@simonrussell so if I understood it correct, you are looking only for ability to restore the cache and do nothing else with this action.

Is my understanding correct?

kotewar avatar Dec 02 '22 08:12 kotewar

@kotewar pretty much -- depending on what branch we're on, so some runtime level of control over whether it goes and updates the cache would be good.

simonrussell avatar Dec 02 '22 08:12 simonrussell

Hey all, 👋🏽

We have created a discussion with a proposal that we feel will solve this problem, do let us know your feedback, thanks 😊

kotewar avatar Dec 07 '22 12:12 kotewar

We have released a new restore action that solves this issue, hence closing it.

kotewar avatar Dec 27 '22 15:12 kotewar