Cache decompression slow on Windows

[craffael]

For me decompression of a cache on Windows is much slower than on linux. A 360Mb cache takes

• only 10sec to download
• but 5 minutes to decompress!

The cache consists of many small files (e.g. the boost library) which probably affects performance negatively.

It seems that appveyor switched to 7zip to compress/decompress the cache exactly for performance reasons. Maybe this would also be on option for github actions that run on windows?

Also note that there was already a similar issue (#336) but it was fixed by improving network performance. Here the bottleneck is not the network but the decompression itself...

[Leseratte10]

I noticed that issue as well. I thought I could use the cache so I don't have to download the needed compilers and libraries for every run, but loading and extracting them from the cache takes way longer than just redownloading them. That makes the cache pretty useless for my use case.

[quanglam2807]

I also have this problem. I run the same job on Linux, macOS & Windows. And while macOS & Linux take less than 1 minutes, Windows takes almost 4 minutes.

[arvinxx]

Yes, I meet this problem too, the Windows cost almost 5mins every cache. It really a time waste !!!

[airglow923]

Experienced the same issue on Windows. Here's the config I used:

- name: Get yarn cache directory
      id: yarn-cache-dir-path
      run: echo "::set-output name=dir::$(yarn cache dir)"

- name: Query yarn cache
      uses: actions/cache@v2
      id: yarn-cache
      with:
        path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
        key: ${{ runner.os }}-yarn-${{ matrix.node-version }}-${{ hashFiles('**/yarn.lock') }}

On ubuntu-lastest:

On windows-latest:

As you can see, that margin is extreme.

[aminya]

I have experienced the same issue. Setting up a cache with a size of 800 MB takes 20 seconds on Ubuntu while the smaller 600 MB cache takes about 1 min.

They use different commands which can be the reason for the slow decompression. I wonder if actions should use 7z on Windows instead of tar.

/usr/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/123/cache.tzst -P -C /home/runner/work/proj/proj

C:\Windows\System32\tar.exe -z -xf D:/a/_temp/123/cache.tgz -P -C D:/a/proj/proj

[aminya]

They have disabled zstd compression on Windows, and that is why Windows caching is so slow. Related to #301

https://github.com/actions/toolkit/blob/439eaced077b820cbf01183f71a220813514a95b/packages/cache/src/internal/cacheUtils.ts#L88-L91

[coreysharris]

Seeing the same thing. Here using actions/cache@v2 on windows-latest (911 MB in 232 secs):

Mon, 14 Jun 2021 19:09:23 GMT Cache Size: ~911 MB (954772818 B)
Mon, 14 Jun 2021 19:09:23 GMT C:\Windows\System32\tar.exe -z -xf D:/a/_temp/43c0bd85-e872-49a2-8d1d-79821cfaf45d/cache.tgz -P -C D:/a/...
Mon, 14 Jun 2021 19:13:15 GMT Cache restored successfully

And here on ubuntu-18.04 (934 MB in 19 secs):

Mon, 14 Jun 2021 19:08:58 GMT Cache Size: ~934 MB (979124632 B)
Mon, 14 Jun 2021 19:08:58 GMT /bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/cb044a1c-0a29-4b6c-a0d5-34ac07f43d8c/cache.tzst -P -C /home/runner/work/...
Mon, 14 Jun 2021 19:09:17 GMT Cache restored successfully

[shichen85]

Also having a very slow cache restore operation for unzipped gradle-7.0.2-all.zip contents at the path C:\Users\runneradmin\.gradle\wrapper\dists\gradle-7.0.2-all on a Windows runner. Downloading the zip file and decompressing it takes under 2 minutes, whereas using cache restore would take up to 3m43s.

[Safihre]

I am seeing the same thing, it's just too slow! Can the team not do something about this? Just pinging some recently active devs @vsvipul, maybe you can help?

[Safihre]

Just as an update: I just switched to runs-on: windows-2022 and suddenly the decompression times are 4x faster. Still slower than Linux/macOS, but much improved. Maybe other see the same? But: still too slow!

[mattjohnsonpint]

@Safihre - Still to slow for me, even on windows-2022. See https://github.com/actions/cache/issues/752#issuecomment-1119931553

[Safihre]

Yeah, I also saw it in my runs. But based on the lack of any responses here, they (Github) don't seem to care at all.

[lvpx]

Hi @shichen85 @coreysharris @Safihre . As @aminya observed, zstd is disabled on Windows due issues with bsd tar. We are working on a longer term solution. You could meanwhile use the workaround suggested here to improve performance. This would enable zstd while using GNU tar.

TLDR; Add the following step to your workflow before the cache step. That's it.

- if: ${{ runner.os == 'Windows' }}
      name: Use GNU tar
      shell: cmd
      run: |
        echo "Adding GNU tar to PATH"
        echo C:\Program Files\Git\usr\bin>>"%GITHUB_PATH%"

Hope this helps.

[Safihre]

For my project this makes it even slower.. Taking 1:20 min to unpack 35MB 😑 Anyone else having better results?

[lvpx]

@Safihre could you share your workflow file or the sample run or logs if possible?

[Safihre]

@pdotl we use an action that caches a Python virtual environment, but under the hood that uses the actions/cache I repeated the workflow run 3 times, so 2 times it had a cache hit. The correct tar seems to be used: https://github.com/sabnzbd/sabnzbd/runs/7958313964?check_suite_focus=true#step:6:23

[lvpx]

@Safihre I'm not sure what the issue is. I tried running a similar albeit simpler workflow to yours using the syphar/[email protected] action but still found windows to be reasonably fast for ~24 mb cache. https://github.com/pdotl/sample-cache-demo/actions/runs/2906664955. Deeper look is needed.

[lvpx]

We are tracking this issue in #984. Please check the current proposal there and provide comments/feedback if any. Closing this as duplicate.