actions
Github-hosted runner-generated cache entry not found from self-hosted runner using action (though findable through REST API)
I run workflows using both Github-hosted runners and self-hosted runners. When a Github-hosted runner stores an artifact in the cache, the self-hosted runner does not find it, even though it is using exactly the same key. I printed out the binary representation of both keys and confirmed they are identical (no hidden characters, etc).
In fact, from the self-hosted runner: if I use curl to search the cache for the key using the REST API, it does find the entry. It just doesn't find it through the action.
The same is true in the other direction, as well (a Github-hosted runner does not get a cache hit for a cache entry with the correct key if it was generated by a self-hosted runner).
The below example is from when the cache entry was added by a Github-hosted runner and cannot be found using actions/cache by the self-hosted runner, and yet it can be found from the same self-hosted runner workflow using curl and the REST API:
Here's the workflow step where I query the cache with the REST API from the self-hosted runner:
- name: Debug cache API lookup
shell: bash
run: |
# recompute the same key you use for actions/cache
python_version=$(cat .python-version)
CACHE_KEY="${RUNNER_OS}-${python_version}-${{ hashFiles('pyproject.toml','setup.cfg','requirements.txt') }}"
echo "Looking up cache key: $CACHE_KEY"
# call the public REST endpoint with GITHUB_TOKEN
curl -fsSL \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
"https://api.github.com/repos/${{ github.repository }}/actions/caches?key=${CACHE_KEY}" \
| jq .
and here is the output from the self-hosted runner workflow:
Run # recompute the same key you use for actions/cache
Looking up cache key: Linux-3.12.3-b7dbd3ea392a6a23acbfac7fae36dded9622e49c3fa5824b0b[16](my_step_and_line_number)my_hash
{
"total_count": 1,
"actions_caches": [
{
"id": my_id,
"ref": "refs/heads/main",
"key": "Linux-3.12.3-b7dbd3ea392a6a23acbfac7fae36dded9622e49c3fa5824b0b168cfa64a66cc1",
"version": "my_version",
"last_accessed_at": "my_info",
"created_at": "more_info",
"size_in_bytes": 2883415849
}
]
}
One of the next steps is:
- name: Python venv caching
uses: actions/[email protected]
id: cache-venv
with:
path: ${{ env.VENV_CACHE }}
key: ${{ runner.os }}-${{ steps.get-python-version.outputs.python_version }}-${{ hashFiles('pyproject.toml', 'setup.cfg', 'requirements.txt') }}
Here is the output from this (on the self-hosted runner):
Run actions/[email protected]
Cache not found for input keys: Linux-3.12.3-b7dbd3ea392a6a23acbfac7fae36dded9622e49c3fa5824b0b168cfa64a66cc1
Note: this is using actions/cache version 4.2.3.
Both the self-hosted runner and the Github-hosted runner are running Ubuntu 24.04 LTS.
BTW, these are the ascii values for the two keys (github-hosted and self-hosted). As you can see, they are identical:
00000000 4c 69 6e 75 78 2d 33 2e 31 32 2e 33 2d 62 37 64 |Linux-3.12.3-b7d| 00000010 62 64 33 65 61 33 39 32 61 36 61 32 33 61 63 62 |bd3ea392a6a23acb| 00000020 66 61 63 37 66 61 65 33 36 64 64 65 64 39 36 32 |fac7fae36dded962| 00000030 32 65 34 39 63 33 66 61 35 38 32 34 62 30 62 31 |2e49c3fa5824b0b1| 00000040 36 38 63 66 61 36 34 61 36 36 63 63 31 |68cfa64a66cc1|
00000000 4c 69 6e 75 78 2d 33 2e 31 32 2e 33 2d 62 37 64 |Linux-3.12.3-b7d| 00000010 62 64 33 65 61 33 39 32 61 36 61 32 33 61 63 62 |bd3ea392a6a23acb| 00000020 66 61 63 37 66 61 65 33 36 64 64 65 64 39 36 32 |fac7fae36dded962| 00000030 32 65 34 39 63 33 66 61 35 38 32 34 62 30 62 31 |2e49c3fa5824b0b1| 00000040 36 38 63 66 61 36 34 61 36 36 63 63 31 |68cfa64a66cc1|
We ended up replacing pip with uv and removed the use of
actions/cache for caching our venv.
On Fri, Nov 7, 2025 at 2:49 AM Manuel Zubieta @.***> wrote:
iautom8things left a comment (actions/cache#1595) https://github.com/actions/cache/issues/1595#issuecomment-3501838225
@bill-taut https://github.com/bill-taut Did you ever find a remediation for this?
— Reply to this email directly, view it on GitHub https://github.com/actions/cache/issues/1595#issuecomment-3501838225, or unsubscribe https://github.com/notifications/unsubscribe-auth/BEKMORMBJVJFKNKA3BBNU6T33R2J7AVCNFSM6AAAAAB3L3X5HKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKMBRHAZTQMRSGU . You are receiving this because you were mentioned.Message ID: @.***>
I ended up figuring out our underlying cause, and apparently other tickets are already opened covering the topic: #1619
We use https://github.com/actions/actions-runner-controller and we were seeing the issue that caches saved on our self-hosted runners could not be restored on the github-hosted runners. I created a new runner image that adds zstd and switched to using that and things work again. It looks like their docker image for ubuntu 20.04 used to have it installed but the two newer images don't. I may see if I can open a PR over there so I can stop using this stop-gap monkey patch. 🙃
Thanks for the response, @bill-taut! 🍻 Cheers