cache icon indicating copy to clipboard operation
cache copied to clipboard

actions/cache and actions/cache/save consider cache to be uploaded sucessfully even with backend errors.

Open caquino opened this issue 1 year ago • 2 comments

Hi,

We are seeing 503 responses from the cloud caching backend (which we reported already to Github support), but the cache actions exit with success when the cache fails to upload, which surfaces later on the workflow as an error while restoring the cache and making harder to troubleshoot the workflow as the root cause of the issue is not the restore, but the save.

Here it is one example of a failure we saw.

##[debug]File Size: 246805853
##[debug]Reserving Cache
##[debug]Resource Url: https://acghubeus1.actions.githubusercontent.com/<INFO HIDDEN>/_apis/artifactcache/caches
##[debug]Saving Cache (ID: 2274)
##[debug]Upload cache
##[debug]Resource Url: https://acghubeus1.actions.githubusercontent.com/<INFO HIDDEN>/_apis/artifactcache/caches/<INFO HIDDEN>
##[debug]Upload concurrency: 4
##[debug]Upload chunk size: 33554432
##[debug]Awaiting all uploads
##[debug]Uploading chunk of size 33554432 bytes at offset 0 with content range: bytes 0-33554431/*
##[debug]Uploading chunk of size 33554432 bytes at offset 33554432 with content range: bytes 33554432-67108863/*
##[debug]Uploading chunk of size 33554432 bytes at offset 67108864 with content range: bytes 67108864-100663295/*
##[debug]Uploading chunk of size 33554432 bytes at offset 100663296 with content range: bytes 100663296-134217727/*
##[debug]uploadChunk (start: 67108864, end: 100663295) - Attempt 1 of 2 failed with error: Cache service responded with 503
##[debug]uploadChunk (start: 100663296, end: 134217727) - Attempt 1 of 2 failed with error: Cache service responded with 503
##[debug]uploadChunk (start: 33554432, end: 67108863) - Attempt 1 of 2 failed with error: Cache service responded with 503
##[debug]uploadChunk (start: 0, end: 33554431) - Attempt 1 of 2 failed with error: Cache service responded with 503
##[debug]uploadChunk (start: 67108864, end: 100663295) - Attempt 2 of 2 failed with error: Cache service responded with 503
Warning: Failed to save: uploadChunk (start: 67108864, end: 100663295) failed: Cache service responded with 503
Cache saved with key: Linux-9315318781-40-2-build
##[debug]Node Action run completed with exit code 0
##[debug]Finishing: Run actions/cache/save@v4

Is it possible to force the action to fail? If not is this something that can be improved?

Thanks

caquino avatar May 31 '24 14:05 caquino


name: Example Workflow

on:
  push:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      # Your build steps here
      - name: Build something
        run: echo "Building..."

      # Cache save step
      - name: Save cache
        id: cache-save
        uses: actions/cache/save@v4
        with:
          path: |
            ~/my-cache-dir
          key: Linux-${{ github.run_id }}-build
          save-always: true  # Ensures the save attempt happens even if prior steps fail

      # Check if cache save failed by looking for signs of success
      # Since there's no direct output, we can use a follow-up step to validate
      - name: Verify cache save
        run: |
          # Attempt to restore the cache immediately to verify it was saved
          # If restore fails, it indicates the save likely failed
          if ! gh api -H "Accept: application/vnd.github+json" \
            "/repos/${{ github.repository }}/actions/caches?key=Linux-${{ github.run_id }}-build" | grep -q "cache_id"; then
            echo "Cache save likely failed (cache not found in API)."
            exit 1
          fi
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

ljluestc avatar Mar 12 '25 15:03 ljluestc

On Wednesday, March 12, 2025 at 11:39:33 AM EDT, ljluestc ***@***.***> wrote:  

name: Example Workflow

on: push: branches: - main

jobs: build: runs-on: ubuntu-latest steps: - name: Checkout code uses: @.***

  # Your build steps here
  - name: Build something
    run: echo "Building..."

  # Cache save step
  - name: Save cache
    id: cache-save
    uses: ***@***.***
    with:
      path: |
        ~/my-cache-dir
      key: Linux-${{ github.run_id }}-build
      save-always: true  # Ensures the save attempt happens even if prior steps fail

  # Check if cache save failed by looking for signs of success
  # Since there's no direct output, we can use a follow-up step to validate
  - name: Verify cache save
    run: |
      # Attempt to restore the cache immediately to verify it was saved
      # If restore fails, it indicates the save likely failed
      if ! gh api -H "Accept: application/vnd.github+json" \
        "/repos/${{ github.repository }}/actions/caches?key=Linux-${{ github.run_id }}-build" | grep -q "cache_id"; then
        echo "Cache save likely failed (cache not found in API)."
        exit 1
      fi
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***> ljluestc left a comment (actions/cache#1416) name: Example Workflow

on: push: branches: - main

jobs: build: runs-on: ubuntu-latest steps: - name: Checkout code uses: @.***

  # Your build steps here
  - name: Build something
    run: echo "Building..."

  # Cache save step
  - name: Save cache
    id: cache-save
    uses: ***@***.***
    with:
      path: |
        ~/my-cache-dir
      key: Linux-${{ github.run_id }}-build
      save-always: true  # Ensures the save attempt happens even if prior steps fail

  # Check if cache save failed by looking for signs of success
  # Since there's no direct output, we can use a follow-up step to validate
  - name: Verify cache save
    run: |
      # Attempt to restore the cache immediately to verify it was saved
      # If restore fails, it indicates the save likely failed
      if ! gh api -H "Accept: application/vnd.github+json" \
        "/repos/${{ github.repository }}/actions/caches?key=Linux-${{ github.run_id }}-build" | grep -q "cache_id"; then
        echo "Cache save likely failed (cache not found in API)."
        exit 1
      fi
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

xzpix avatar Jun 10 '25 08:06 xzpix