setup-node icon indicating copy to clipboard operation
setup-node copied to clipboard

Caching fallback if there is no exact hit

Open dbartholomae opened this issue 4 years ago • 4 comments

Hi there,

at the moment, caching only hits if the exact same lockfile exists, but most of the time, even it the lockfile slightly changes, it is still beneficial to restore cache instead of downloading everything from scratch (most likely only ~1 dependency changed). I would propose that, by default, cache falls back to node-cache-${platform}-${packageManager} instead of only checking node-cache-${platform}-${packageManager}-${fileHash}. If this is of interest, I'm happy to provide a PR. It seems to only need a test and a one line change in https://github.com/actions/setup-node/blob/main/src/cache-restore.ts#L44

See also https://github.com/actions/setup-node/issues/304#issuecomment-904553549

dbartholomae avatar Sep 05 '21 16:09 dbartholomae

There seem to be implementations for this in #323 and https://github.com/thrivehealth/setup-node/commit/bd6483add60213c255e1f5297f6394ed26df706c

oxc avatar Feb 01 '22 10:02 oxc

Just for sharing some workarounds with @action/cache

  • With Pnpm 7+: https://gist.github.com/belgattitude/838b2eba30c324f1f0033a797bab2e31
  • With Yarn 3+: https://gist.github.com/belgattitude/042f9caf10d029badbde6cf9d43e400a

belgattitude avatar Oct 10 '22 21:10 belgattitude

New PR to fix this: https://github.com/actions/setup-node/pull/702

voxpelli avatar Feb 27 '23 18:02 voxpelli

Rebased and updated the PR over at #702, needed adaption after #744 was merged

voxpelli avatar Mar 13 '24 20:03 voxpelli

Hello Everyone, Thank you for raising this request and for the thoughtful PR. We completely understand the motivation behind introducing a fallback caching mechanism, as it could help improve efficiency in certain scenarios.

That said, after discussion, we’ve decided not to move forward with this change. The main concern is that fallback caching may lead to unpredictable behavior like restoring stale dependencies, introducing subtle inconsistencies, and making workflows harder to debug. Ensuring reproducibility and determinism is a core design principle for setup-node, and strict cache key matching helps guarantee that.

For teams that prefer more flexible caching strategies, we recommend using actions/cache directly, where you can design custom keys and fallbacks as needed.

We really appreciate your contribution and the effort put into this proposal. Even though we can’t merge this change, we encourage you to continue sharing ideas and feedback with us, as it helps shape the future improvements.

aparnajyothi-y avatar Sep 30 '25 10:09 aparnajyothi-y

Can you say more about the specific "subtle inconsistencies" and "stale dependencies" you're worried about? At least with Yarn, my understanding is that it'll only pull things from the disk cache if they're an exact match with the lockfile and if the checksum is correct (see checksumBehavior on https://yarnpkg.com/configuration/yarnrc).

I think it also makes intuitive sense that package managers will function correctly with a cache that doesn't contain exactly what it needs. Consider the case where I clone a random project to my machine and run yarn install or similar: a package manager will use the cache if it can, but it'll still happily pull anything that's missing from the cache from the npm registry.

I'd appreciate links to specific GitHub issues or sections of package manager documentation that explains how they could possibly end up constructing an incorrect dependency tree if the cache is "outdated".

I should also add that using a fallback restore key matches GitHub's own best practices as documented in actions/cache: https://github.com/actions/cache/blob/main/examples.md#node---yarn

nwalters512 avatar Sep 30 '25 14:09 nwalters512