Could not get aws public registry endpoint

[jeff51419]

Describe the bug could not get image from public.ecr.aws registry.

To Reproduce

1. create argocd-image-updater
2. kubectl exec -it argocd-image-updater-bb9c65fbd-rdfgn -- sh
3. $ argocd-image-updater test public.ecr.aws/nginx/nginx

INFO[0000] getting image                                 image_name=nginx/nginx registry=public.ecr.aws
FATA[0000] could not get registry endpoint: no registry with prefix 'public.ecr.aws' configured

---

Following the [Configuring Container Registries](https://argocd-image-updater.readthedocs.io/en/v0.9.0/configuration/registries/#configuring-container-registries)

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: argocd-image-updater-config
    app.kubernetes.io/part-of: argocd-image-updater
  name: argocd-image-updater-config
data:
  registries.conf: |
    registries:
    - name: Public amazon ECR Hub
      api_url: https://public.ecr.aws
      prefix: public.ecr.aws
      ping: no
      tagsortmode: latest-first
      credsexpire: 6h

verified in pod

Expected behavior argocd-image-updater can access public.ecr.aws, then update the images.

Additional context aws doc https://docs.aws.amazon.com/AmazonECR/latest/public/public-registries.html#public-registry-auth

Version $ argocd-image-updater version

argocd-image-updater: v0.9.3+06cbc09
  BuildDate: 2021-05-05T07:24:04Z
  GitCommit: 06cbc097ae3b7a77458ba657d0ef7e3028477a54
  GoVersion: go1.14.13
  GoCompiler: gc
  Platform: linux/amd64

Logs pod log

If there is anything I set up in wrong way, please let me know.

[jannfis]

Hey @jeff51419,

for the argocd-image-updater test command, you will have to explicitly pass the registries.conf to use if you have defined a custom registry, e.g.:

$ cat registries.conf 
registries:
- name: Public amazon ECR Hub
  api_url: https://public.ecr.aws
  prefix: public.ecr.aws
  ping: no
  tagsortmode: latest-first
  credsexpire: 6h
$ ./dist/argocd-image-updater test public.ecr.aws/nginx/nginx --registries-conf registries.conf 
INFO[0000] getting image                                 image_name=nginx/nginx registry=public.ecr.aws
DEBU[0000] rate limit for https://public.ecr.aws is 2147483647 
INFO[0000] Loaded 1 registry configurations from registries.conf 
INFO[0000] Fetching available tags and metadata from registry  image_name=nginx/nginx
FATA[0001] could not get tags: Get "https://public.ecr.aws/v2/nginx/nginx/tags/list": http: non-successful response (status=404 body="404 page not found\n") 

Apparently, despite named as public, the ECR requires authentication for the HTTP API according to https://docs.aws.amazon.com/AmazonECR/latest/public/public-registries.html.

Unfortunately, I do not have an aws account to verify this, but I believe you will have to set up authentication in order to use public.ecr.aws.

[jeff51419]

hi @jannfis

Thanks for your information. So i need to login AWS account in "argocd-image-updater" pod, right? I will test it later.

[stobias123]

AWS ECR doesn't support list it seems https://github.com/aws/containers-roadmap/issues/1262

However.... Private does. but we need to include bearer token... docs don't make it clear how that could be possible.

https://docs.aws.amazon.com/AmazonECR/latest/userguide/registry_auth.html

[jlbutler]

Hi all,

I have made a https://github.com/aws/containers-roadmap/issues/1262#issuecomment-1213127749 tracking tags missing from ECR Public's v2 API.

The quick ask for feedback: if we iterate toward full OCI compliance, for the ArgoCD Image Updater use case is it OK for us to consider deferring the lexical sorting and transparent pagination token aspect of the spec, to focus on a first delivery that is the correct response body? We're very interested in unblocking use cases like this primarily.

@jannfis and others, if you have the time, can you reply here or on the roadmap issue? And, thanks for your time!

[jlbutler]

However.... Private does. but we need to include bearer token... docs don't make it clear how that could be possible.

https://docs.aws.amazon.com/AmazonECR/latest/userguide/registry_auth.html

You can retrieve an ECR auth token via API or CLI. They do expire after 12h, but Image Updater has config to refresh.

See the note under "Specify credential expiry time in the registry configuration" here in the docs

Hope that helps!

[jannfis]

@jlbutler, glad to see that aws actually is trying to improve their public registry! Thanks for finding this issue, and asking for feedback!

is it OK for us to consider deferring the lexical sorting and transparent pagination token aspect of the spec, to focus on a first delivery that is the correct response body?

Pagination is optional from Image Updater's point of view, as is lexical sorting of the tags. So that should be fine for us, as long as we can retrieve the metadata of the images by performing layer pulls.

[jlbutler]

@jannfis thanks so much for the reply. we are testing the tags/list implementation now, and we'll look to iterate on ordering and the pagination token transparency.

i believe everything else should be there in the /v2 api. if you can share a bit of detail on the metadata retrieval and what's needed, i can do some work to ensure it'll meet needs.

thanks again!

[jannfis]

@jbutler We use the official Docker reference API (https://github.com/distribution/distribution) under the hood to perform pulling of manifests from a given image repository, to get details such as build dates and digests.

There's actually a very simple method to test whether your API will be usable by argocd-image-updater. You can grab a binary from the releases page and then:

1. To only fetch tag list (no metadata is retrieved)

argocd-image-updater test --update-strategy semver public.ecr.aws/some/image

2. To also fetch some metadata:

argocd-image-updater test --update-strategy latest public.ecr.aws/some/image

More information here: https://argocd-image-updater.readthedocs.io/en/stable/install/testing/

Happy to also help out with testing, if required!