kraken icon indicating copy to clipboard operation
kraken copied to clipboard

500 failed to verify data: digests do not match; Image fails to pull.

Open will-ds opened this issue 3 years ago • 1 comments

Describe the bug Pulling an image (whether it be the origin, agent or manually) returns the following in the origin logs (/var/log/kraken/kraken-origin/) after re-deploying kraken. Image fails to pull.

INFO	handler/handler.go:98	500 GET /namespace/nvidia/cuda/blobs/sha256:2806d80fe28f9a42dfe4a1b30aeaefb3f75436d3c7fbfc6dfe7e71e1e5d3840a failed to verify data: digests do not match

To Reproduce Steps to reproduce the behavior:

  1. Spin up Kraken using the helm chart. Configure GCS as the backend.
  2. Push an image via the kraken proxy.
  3. Run a separate deployment that uses this image. Verify it gets pulled and runs correctly.
  4. Tear down both deployments.
  5. Re-deploy kraken using helm chart using same configuration as Step 1.
  6. Run a new deployment using the same yaml file as Step 3.

Check logs of kraken-origin to find the above error.

Expected behavior Image should pull down to origin(s).

Screenshots If applicable, add screenshots to help explain your problem.

Environments

  • Kubernetes on Google Cloud
  • Kraken backend is Google Cloud Storage
  • Running Kraken 0.1.4

Additional context Add any other context about the problem here--environment configurations, logs, etc.

Build index backend configuration:

    - namespace: .*
      backend:
        gcs:
          username: kraken-service-account
          bucket: gke_image_cache
          root_directory: /tags/
          name_path: docker_tag

Oirigin backend configuration:

    - namespace: .*
      backend:
        gcs:
          username: kraken-service-account
          bucket: gke_image_cache
          root_directory: /blobs/
          name_path: identity

will-ds avatar Apr 01 '21 22:04 will-ds

@will-ds I have the same error, did you find any solution? Unfortunately, debug logs don't return any additional info.

ish-xyz avatar Jun 10 '21 10:06 ish-xyz