argocd-image-updater icon indicating copy to clipboard operation
argocd-image-updater copied to clipboard

Cannot pull images from private ECR repository

Open WildEgor opened this issue 2 years ago • 6 comments

Describe the bug ArgoCD image updater cannot pull image. AWS .env below has access to ECR and may pull/push images locally with docker-compose. ArgoCD with updater and application pods deployed in different namespaces. You can you this repo as reference: https://github.com/WildEgor/argocd-boilerplate

To Reproduce

  1. Install minikube;
  2. Install ArgoCD (using helm);
  3. Add own repo with Application-s;
  4. Install ArgoCD Image Updater (using helm) with these values:
config:
  registries:
    - name: Docker Hub
      api_url: https://registry-1.docker.io
      prefix: docker.io
    - name: ECR
      api_url: https://***.dkr.ecr.ap-south-1.amazonaws.com
      prefix: ***.dkr.ecr.ap-south-1.amazonaws.com 
      default: true 
      ping: yes
      tagsortmode: none
      insecure: no
      credentials: ext:/scripts/ecr-login.sh 
      credsexpire: 6h

authScripts:
  enabled: true
  scripts:
    ecr-login.sh: |   # notice script name matches above    
      #!/bin/sh
      aws ecr --region $AWS_REGION get-authorization-token --output text --query 'authorizationData[].authorizationToken' | base64 -d

extraEnv:
  - name: AWS_REGION
    value: "ap-south-1"
  - name: AWS_ACCESS_KEY_ID
    value: "****"
  - name: AWS_SECRET_ACCESS_KEY
    value: "****"

serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::***:role/***
  name: "argocd-image-updater"
  1. Find argocd image updater pod and see logs

Expected behavior Successfully pull images from private ECR

Version Windows 10 Minikube v1.28.0 Docker 20.10.22, build 3a2c30b Kubernetes 1.25

Logs

time="2023-01-29T11:15:46Z" level=info msg="Starting image update cycle, considering 2 annotated application(s) for update"
time="2023-01-29T11:15:48Z" level=error msg="Could not get tags from registry: Get \"https://***.dkr.ecr.***.amazonaws.com/v2/***/tags/list\": no basic auth credentials" alias=*** application*** image_name=*** image_tag=latest registry=***.dkr.ecr.***.amazonaws.com
time="2023-01-29T11:15:48Z" level=error msg="Could not get tags from registry: Get \"https://***.dkr.ecr.***.amazonaws.com/v2/***/tags/list\": no basic auth credentials" alias=gateway application=*** image_name=*** image_tag=latest registry=***.dkr.ecr.***.amazonaws.com
time="2023-01-29T11:15:48Z" level=info msg="Processing results: applications=2 images_considered=2 images_skipped=0 images_updated=0 errors=2"

After while and some magic things I see commit from argo in my repository (with right sha of my image) and logs:

time="2023-01-29T12:05:12Z" level=info msg="Successfully updated the live application spec" application=***
time="2023-01-29T12:05:12Z" level=info msg="Processing results: applications=2 images_considered=2 images_skipped=0 images_updated=2 errors=0"

But in pods with images still error:

container "common" in pod "***-7b78c5f98d-pnllz" is waiting to start: trying and failing to pull image

Failed to pull image "***.dkr.ecr.ap-south-1.amazonaws.com/***:latest": rpc error: code = Unkno
wn desc = Error response from daemon: Head "https://***.dkr.ecr.ap-south-1.amazonaws.com/v2/***/manifests/latest": no basic auth credentials

WildEgor avatar Jan 29 '23 11:01 WildEgor

Hello i have the same error

clement94310 avatar Apr 18 '23 09:04 clement94310

I initially hit this error as well but following the guidance of this comment and using the helm chart got things working for me.

seanmorton avatar May 18 '23 19:05 seanmorton

Hi @WildEgor and @clement94310 , I have the same error and cannot find anything wrong in configuration. Have you solved your issue?

kyleli666 avatar Jul 20 '23 16:07 kyleli666

Hi @WildEgor and @clement94310 , I have the same error and cannot find anything wrong in configuration. Have you solved your issue?

Unfortunately no

WildEgor avatar Jul 20 '23 18:07 WildEgor

@WildEgor @kyleli666 hi guys any update on this?

zhaque44 avatar Dec 13 '23 21:12 zhaque44