cache icon indicating copy to clipboard operation
cache copied to clipboard

Caches created on default branch, but are missed on PR branches, despite using the same key

Open jbreckmckye opened this issue 2 years ago • 3 comments

I have a problem with GHA's cache action in one of my organisation's monorepos. We are storing caches on the default branch, however PRs off that branch - using the cache action with the same key - are getting cache misses.

Our configuration

The default branch (develop) has a GHA config that can be simplified to:

  • check out the project
  • use actions/cache@v2 with a key of ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('yarn.lock') }}-${{ hashFiles('**/bower.json') }}-${{ hashFiles('.yarn/releases/yarn-*') }}
  • execute Yarn install
  • store the cache
Simplified config yml
name: Sets up common node_modules cache for use in branch builds
on:
  push:
    branches:
      - "develop"

jobs:
  main:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Use Node.js 14.x
        uses: actions/[email protected]
        with:
          node-version: '14.x'
      - name: Configure auth credentials
        { ... }
      - name: Set npm registry auth token
        { ... }
      - name: Cache node modules
        id: cache-node-modules
        uses: actions/cache@v2
        env:
          cache-name: cache-node-modules
        with:
          path: |
            **/node_modules
          key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('yarn.lock') }}-${{ hashFiles('**/bower.json') }}-${{ hashFiles('.yarn/releases/yarn-*') }}
          restore-keys: |
            ${{ runner.os }}-build-${{ env.cache-name }}-
            ${{ runner.os }}-build-
            ${{ runner.os }}-
      - name: Install dependencies
        if: steps.cache-node-modules.outputs.cache-hit != 'true'
        run: yarn install --immutable

In pull requests we have a GHA config that can be simplified to

  • check out the project
  • use actions/cache@v2 with a key of ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('yarn.lock') }}-${{ hashFiles('**/bower.json') }}-${{ hashFiles('.yarn/releases/yarn-*') }}
  • execute Yarn install if cache was missed
Simplified config yml
name: CI

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
    branches:
      - 'develop'

concurrency:
  group: ci-${{ github.head_ref }}
  cancel-in-progress: true

jobs:
  build:
    if: github.event.pull_request.draft == false
    runs-on: ubuntu-18.04
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    { ... }
    - name: Use Node.js 14.x
      uses: actions/[email protected]
      with:
        node-version: '14.x'
    - name: Configure auth credentials
    { ... }
    - name: Set npm auth token
    { ... }
    - name: Cache node modules
      id: cache-node-modules
      uses: actions/cache@v2
      env:
        cache-name: cache-node-modules
      with:
        path: |
          **/node_modules
          src/modules/library/eslint-plugin-feature-modules
        key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('yarn.lock') }}-${{ hashFiles('**/bower.json') }}-${{ hashFiles('.yarn/releases/yarn-*') }}
        restore-keys: |
          ${{ runner.os }}-build-${{ env.cache-name }}-
          ${{ runner.os }}-build-
          ${{ runner.os }}-
    - name: Install dependencies
      if: steps.cache-node-modules.outputs.cache-hit != 'true'
      run: yarn install --immutable

     { ... other steps }

The problem

Even though the job on the default branch (develop) succeeds, and says it's stored a cache (Cache saved successfully), and the last success was 2 days ago (so it's not old enough to evict) - new PRs with the same cache key get a cache miss.

For example, here's the output of the default branch job:

Post job cleanup.
/usr/bin/tar --posix --use-compress-program zstd -T0 -cf cache.tzst -P -C /home/runner/work/gousto-webclient/gousto-webclient --files-from manifest.txt
Cache Size: ~435 MB (456174300 B)
Cache saved successfully
Cache saved with key: Linux-build-cache-node-modules-6c6b94ae2431351658c69d35ff5da10d23b521ff39852819585aa8fddeb9ed0b-be927375992e824a205144337571c97f7dd52c144bccdce88e6649e9062c5dcb-3e44a4a4e1284aa53e6892051391472357fece638301544521d433977ddbbd87

And here's the output of a new PR, just created today. This branch was taken off the default branch and merges back into it.

Cache not found for input keys: Linux-build-cache-node-modules-6c6b94ae2431351658c69d35ff5da10d23b521ff39852819585aa8fddeb9ed0b-be927375992e824a205144337571c97f7dd52c144bccdce88e6649e9062c5dcb-3e44a4a4e1284aa53e6892051391472357fece638301544521d433977ddbbd87, Linux-build-cache-node-modules-, Linux-build-, Linux-

Request

  • Can I confirm that this is not expected behaviour?
  • Can I rule out any configuration issues on my side that could be breaking the caching? Are there any APIs or debugging tools I can use to help understand what's wrong?
  • Are there any known bugs with the cache behaviour?

jbreckmckye avatar Sep 14 '22 13:09 jbreckmckye

At a quick glance I can see that "path" is different between the two workflows. Cache uniqueness depends on path apart from key.

bishal-pdMSFT avatar Sep 14 '22 15:09 bishal-pdMSFT

Ah - thanks - you have a keener eye than me. This appears to be a new addition to our repository. Let me see if mirroring the path values restores the cache recovery.

jbreckmckye avatar Sep 14 '22 17:09 jbreckmckye

Ah - thanks - you have a keener eye than me. This appears to be a new addition to our repository. Let me see if mirroring the path values restores the cache recovery.

Did this end up working for you? I'm experiencing the same issue but adding on more paths didn't resolve the issue.

mapipina avatar Oct 14 '22 15:10 mapipina

@mapipina Yes, this fixed my issue.

jbreckmckye avatar Nov 01 '22 17:11 jbreckmckye