Proxy cache list tags from remote works well but pull image failed

[chlins]

What can we help you?

1. setup registry of dockerhub with personal auth in harbor
2. push private image to dockerhub
3. create public proxy cache type project in harbor
4. curl harbor.dev/v2/proxy/chlins/nginx/tags/list api can list tags successfully
5. docker pull harbor.dev/proxy/chlins/nginx:1.7 faild

$ docker pull harbor.dev/proxy/chlins/nginx:1.7
Error response from daemon: unknown: artifact proxy/chlins/nginx@sha256:2204e6cc1bbcc3c5587aaada2c59819599847f1dbe5876dc21eba6764aa795ef not found

core.log

2022-02-28T02:29:32Z [WARNING] [/server/middleware/repoproxy/proxy.go:143]: Artifact: proxy/chlins/nginx:1.7, digest: is not found in proxy cache, fetch it from remote repo
2022-02-28T02:29:35Z [ERROR] [/server/middleware/repoproxy/proxy.go:103]: failed to proxy manifest, fallback to local, request uri: /v2/proxy/chlins/nginx/manifests/sha256:2204e6cc1bbcc3c5587aaada2c59819599847f1dbe5876dc21eba6764aa795ef, error: login to dockerhub error: Post "https://hub.docker.com/v2/users/login/": EOF
2022-02-28T02:32:07Z [WARNING] [/server/middleware/repoproxy/proxy.go:143]: Artifact: proxy/chlins/nginx:1.7, digest: is not found in proxy cache, fetch it from remote repo
2022-02-28T02:32:10Z [ERROR] [/server/middleware/repoproxy/proxy.go:103]: failed to proxy manifest, fallback to local, request uri: /v2/proxy/chlins/nginx/manifests/sha256:2204e6cc1bbcc3c5587aaada2c59819599847f1dbe5876dc21eba6764aa795ef, error: login to dockerhub error: Post "https://hub.docker.com/v2/users/login/": EOF
2022-02-28T02:35:15Z [WARNING] [/server/middleware/repoproxy/proxy.go:143]: Artifact: proxy/chlins/nginx:1.7, digest: is not found in proxy cache, fetch it from remote repo
2022-02-28T02:35:17Z [ERROR] [/server/middleware/repoproxy/proxy.go:103]: failed to proxy manifest, fallback to local, request uri: /v2/proxy/chlins/nginx/manifests/sha256:2204e6cc1bbcc3c5587aaada2c59819599847f1dbe5876dc21eba6764aa795ef, error: login to dockerhub error: Post "https://hub.docker.com/v2/users/login/": EOF

[chlins]

The issue maybe posed by dockerhub network jitter.

[chlins]

Reproduce when reach dockerhub rate limit, should wrap root cause to error log for docker client.

2022-03-09T06:41:32Z [WARNING] [/server/middleware/repoproxy/proxy.go:155]: Proxy to remote failed, fallback to local repo, error: http status code: 429, body: {
  “errors”: [
    {
      “code”: “TOOMANYREQUESTS”,
      “message”: “You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit”
    }
  ]
}

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.

[stonezdj]

I am trying to fix it

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.