amazon-ecr-credential-helper icon indicating copy to clipboard operation
amazon-ecr-credential-helper copied to clipboard

Don't give already expired cert when failing to retrieve new tokens

Open ashi009 opened this issue 1 year ago • 0 comments

cache.AuthEntry.IsValid returns false long before the token is actually expired, and triggers cli.defaultClient.GetCredentialsByRegistryID() to fetch a new token with c.getAuthorizationToken(). When c.getAuthorizationToken() failed the token from cache will be returned even if it's actually expired.

https://github.com/awslabs/amazon-ecr-credential-helper/blob/b6714ac5eb132d3d3493b8d660d59a6e4a5aee6e/ecr-login/api/client.go#L154C1-L155C1

Which result in 403s from AWS service, and the logging message is difficult to parse:

Unable to publish images: unable to push image NNNN.dkr.ecr.us-west-2.amazonaws.com/blah: HEAD https://NNNN.dkr.ecr.us-west-2.amazonaws.com/blah/blobs/sha256:...: unexpected status code 403 Forbidden (HEAD responses have no body, use GET for details)

And we can find the error message from the ecr-login logs afterwards:

time="2023-11-21T00:21:32Z" level=debug msg="Checking file cache" registry=NNNN time="2023-11-21T00:21:32Z" level=debug msg="Cached token is no longer valid" expiresAt="2023-11-17 07:24:04.14 +0000 UTC" requestedAt="2023-11-16 19:24:04.151845422 +0000 UTC" time="2023-11-21T00:21:32Z" level=debug msg="Calling ECR.GetAuthorizationToken" registry=NNNN time="2023-11-21T00:21:32Z" level=info msg="Got error fetching authorization token. Falling back to cached token." error="ecr: Failed to get authorization token: not found, ResolveEndpointV2"

ashi009 avatar Nov 22 '23 04:11 ashi009