buildkit-cache-dance icon indicating copy to clipboard operation
buildkit-cache-dance copied to clipboard

Having separate inject steps causes caches to be overwritten

Open Redmega opened this issue 1 year ago • 3 comments

I have two separate caches (one for dependencies, and one for next build caches within a monorepo) that I want to use in my docker build.

The problem is the cache-dance action seems to use the same source folder when extracting the caches. When I added my next build cache to the workflow, it seems that now it overwrites the mounted directed from the first inject step and now the yarn dependencies do not get detected by the docker build.

If I combine all my caches together I'll lose the ability to skip extraction on the dependencies which change much less frequently. I will waste time extracting my yarn cache despite it not changing.

      - name: Set up Yarn build cache
        id: yarn-cache
        uses: actions/cache@v4
        with:
          path: yarn-build-cache
          key: ${{ matrix.platform }}-yarn-${{ hashFiles('yarn.lock') }}
          restore-keys: |
            ${{ matrix.platform }}-yarn-
      - name: Set up Next build cache
        id: next-cache
        uses: actions/cache@v4
        with:
          path: |
            next-build-cache
            nx-build-cache
          key: ${{ matrix.platform }}-next-${{ matrix.app.name }}-${{ hashFiles('yarn.lock') }}-${{ hashFiles(format('apps/{0}/**', matrix.app.name)) }}
          restore-keys: |
            ${{ matrix.platform }}-next-${{ matrix.app.name }}-${{ hashFiles('yarn.lock') }}-
      - name: Inject caches into Docker
        uses: reproducible-containers/buildkit-cache-dance@v3
        with:
          cache-map: |
            {
              "yarn-build-cache": "/fe/.yarn/cache"
            }
          skip-extraction: ${{ steps.yarn-cache.outputs.cache-hit }}
      - name: Inject next cache into Docker
        uses: reproducible-containers/buildkit-cache-dance@v3
        with:
          cache-map: |
            {
              "next-build-cache": "/fe/apps/${{ matrix.app.name }}/.next/cache",
              "nx-build-cache": "/fe/.nx"
            }
          skip-extraction: ${{ steps.next-cache.outputs.cache-hit }}

Redmega avatar Jul 25 '24 00:07 Redmega

I think I'm seeing the same behavior, but unsure. Regardless, it seems that just changing the cache-dance directory such that it's configurable would let this be fixed, or having it be something like /var/hash(paths).

insanitybit avatar Sep 19 '24 19:09 insanitybit

I'm hitting the exact same issue, with the exact same use case. Caching both node_modules and the .next cache means we need them in separate keys. Is the maintainer still around? @Redmega did you figure out a workaround?

anthonyalayo avatar May 08 '25 07:05 anthonyalayo

This was a while ago, I think I tried to cache them together and then access them through subfolders maybe? But ultimately I dropped it and just stopped trying to cache them. The time it took to cache them/load them from cache happened to make the time savings negligible for my use case.

Redmega avatar May 14 '25 17:05 Redmega