On monorepo with lerna & yarn workspaces cache miss every build

[felixmosh]

First of all thank you for this action 🙏🏼 I've noticed that every build of my project the cache always misses. I've compared the restore-key between 2 builds (which yarn.lock didn't changed) and they are the same.

Any pointers?

[felixmosh]

New build

Old build

both of them are mapped to the same cache-key

d7fe8e51825b14d0a67f622d69785219de9a0d33b498c2fb8042f0bf27804c0bd2f37d826feaa07eee82fbd09d963f38e217c68f02eeecc332dbd1a3a9b4ff99

[timbuckley]

@felixmosh Could you post your workflow yaml?

[felixmosh]

name: Build & Deploy
on:
  push:
    tags:
      - 'v1.*'
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
        with:
          node-version: 12.x

      - uses: bahmutov/npm-install@v1

      - name: Build artifacts
        run: yarn build
        env:
          CI: true

      - name: Deploy
        run: yarn shipit production fast-deploy

[felixmosh]

@timbuckley can you help?

[META-DREAMER]

Also encountering this issue, its really slowing down our CI/CD process

[moltar]

Using yarn & lerna here, and it seems to work

https://github.com/berstend/puppeteer-extra

[nealchandra]

Also seeing this issue, would appreciate any insight on how to fix. @moltar doesn't appear to work for you either -- if you look the yarn install step is taking 30-50 sec which shouldn't be the case for a cache hit.

[moltar]

If you are using the same caching systems as GitHub's native system, then it also had a lot of issues in the last few months with fetching data.

Don't know if it's related, but here is their issue: https://github.com/actions/cache/issues/267

[moltar]

Also, if yarn install pays attention to hash of the package file, then this would not use cache in most of my pushes, as I was messing a lot with package.json files.

[davidcalhoun]

I'm also getting yarn install times around 30 seconds with this, which I'm curious if I can improve. I'm getting cache hits (see log below), but it seems to take quite a while to get past the third step ([3/4] Linking dependencies).

Run bahmutov/npm-install@v1
running npm-install GitHub Action
trying to restore cached NPM modules
Received 0 of 118074452 (0.0%), 0.0 MBs/sec
Received 79691776 of 118074452 (67.5%), 38.0 MBs/sec
Received 118074452 of 118074452 (100.0%), 37.6 MBs/sec
Cache Size: ~113 MB (118074452 B)
/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/219d6af3-2420-4046-baaa-91c7e514b062/cache.tzst -P -C /home/runner/work/msat-ui/msat-ui
npm cache hit yarn-linux-x64-4d59f5d0f902bfeacf6649ab8e167c599fad27caa38b3233dcbca93306c5cd74c84eff4f9611fdf03087467c56124fb439380d533f264cb25cccb03525128e46
installing NPM dependencies using Yarn
yarn at "/usr/bin/yarn"
/usr/bin/yarn --frozen-lockfile
yarn install v1.22.4
[1/4] Resolving packages...
[2/4] Fetching packages...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
...
[4/4] Building fresh packages...
Done in 30.59s.
all done, exiting

[davidcalhoun]

Quick update: I was able to improve the performance somewhat by using actions/cache@v2 instead. Using bahmutov/npm-install@v1, was taking 30+ seconds total on a cache hit, which isn't too bad. But using actions/cache@v2, I'm able to restore from the cache in 15-18 seconds.

Here's what my workflow looks like:

# .github/workflows/main.yml
name: Main workflows.
on: push
jobs:
    build-and-test:
        runs-on: ubuntu-latest
        steps:
            - name: Checkout repo.
              uses: actions/checkout@v2
            - name: Set yarn cache directory
              run: echo "::set-env name=YARN_CACHE::$(yarn cache dir)"
            - name: Cache node modules
              id: lerna-yarn-cache
              uses: actions/cache@v2
              env:
                  cache-name: cache-node-modules
              with:
                  path: |
                      ${{ env.YARN_CACHE }}
                      node_modules
                      */*/node_modules
                  key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/yarn.lock') }}
                  restore-keys: |
                      ${{ runner.os }}-build-${{ env.cache-name }}-
                      ${{ runner.os }}-build-
                      ${{ runner.os }}-
            - name: Install dependencies if needed.
              if: steps.lerna-yarn-cache.outputs.cache-hit != 'true'
              run: yarn install --ignore-platform --frozen-lockfile