fix: git write back Helm value files

[burnb]

trafficstars

Related: #650 Fixes for #636 Based on PR #651

• does not consider image aliases
• does not support nested param names
• image name and tag settled in helm value file only for the last image in the annotation image list

[devopsidiot]

Oh I'm cheering for this.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 74.19355% with 8 lines in your changes missing coverage. Please review.

Project coverage is 57.10%. Comparing base (cdb4428) to head (17e9e27).

Files	Patch %	Lines
pkg/argocd/update.go	74.19%	7 Missing and 1 partial :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #663      +/-   ##
==========================================
+ Coverage   57.03%   57.10%   +0.07%     
==========================================
  Files          30       30              
  Lines        2979     3003      +24     
==========================================
+ Hits         1699     1715      +16     
- Misses       1145     1152       +7     
- Partials      135      136       +1     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[devopsidiot]

Hi @burnb - I really appreciate your work on this. I tried this out this morning and unfortunately still ran into the error Could not update application spec: could not find an image-name annotation for image app-test" application=test

[burnb]

Hi @burnb - I really appreciate your work on this. I tried this out this morning and unfortunately still ran into the error Could not update application spec: could not find an image-name annotation for image app-test" application=test

Hi @devopsidiot could you show me what kind of annotation did you try to execute?

[devopsidiot]

Hi @burnb - Here you go (and thanks for responding):

metadata:
  name: daily-worship-test
  namespace: argo
  annotations:
    argocd-image-updater.argoproj.io/image-list: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-tag: develop-v0.20.0-33
    argocd-image-updater.argoproj.io/allow-tags: regexp:develop-v.*
    argocd-image-updater.argoproj.io/update-strategy: newest-build
    argocd-image-updater.argoproj.io/pull-secret: ext:/scripts/ecr-login.sh
    argocd-image-updater.argoproj.io/git-repository: [email protected]:<company>/gitops.git
    argocd-image-updater.argoproj.io/write-back-method: git:secret:argo-image-updater/argocd-image-updater-git-creds
    argocd-image-updater.argoproj.io/write-back-target: "helmvalues:values-rnd.yaml"
    argocd-image-updater.argoproj.io/git-branch: rnd
    ```

[burnb]

Hi @burnb - Here you go (and thanks for responding):

metadata:
  name: daily-worship-test
  namespace: argo
  annotations:
    argocd-image-updater.argoproj.io/image-list: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-tag: develop-v0.20.0-33
    argocd-image-updater.argoproj.io/allow-tags: regexp:develop-v.*
    argocd-image-updater.argoproj.io/update-strategy: newest-build
    argocd-image-updater.argoproj.io/pull-secret: ext:/scripts/ecr-login.sh
    argocd-image-updater.argoproj.io/git-repository: [email protected]:<company>/gitops.git
    argocd-image-updater.argoproj.io/write-back-method: git:secret:argo-image-updater/argocd-image-updater-git-creds
    argocd-image-updater.argoproj.io/write-back-target: "helmvalues:values-rnd.yaml"
    argocd-image-updater.argoproj.io/git-branch: rnd
    ```

helm.image-name and helm.image-tag are the paths to the values which will be generated by updater in the helmvalues file, in your case in the values-rnd.yaml So it could be for example:

argocd-image-updater.argoproj.io/helm.image-name: image.name
argocd-image-updater.argoproj.io/helm.image-tag: image.tag

and it will be represented in values-rnd.yaml after updater refresh like

image:
  name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
  tag: develop-v0.20.0-33

[devopsidiot]

So I did the things correctly, but it's still not updating the application:

level=debug msg="found 32 from 32 tags eligible for consideration" image="<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.20.0-33"
level=info msg="Setting new image to <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.21.0-181" alias= application=daily-worship-test image_name=daily-worship image_tag=develop-v0.20.0-33 registry=<accountid>.dkr.ecr.us-east-1.amazonaws.com
level=debug msg="target parameters: image-spec= image-name=image.name, image-tag=image.tag" application=daily-worship-test image=<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
time="2024-01-16T18:22:45Z" level=info msg="Successfully updated image '<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.20.0-33' to '<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.21.0-181', but pending spec update (dry run=false)" alias= application=daily-worship-test image_name=daily-worship image_tag=develop-v0.20.0-33 registry=<accountid>.dkr.ecr.us-east-1.amazonaws.com
level=debug msg="Using commit message: build: automatic update of daily-worship-test\n\nupdates image daily-worship tag 'develop-v0.20.0-33' to 'develop-v0.21.0-181'\n"
level=info msg="Committing 1 parameter update(s) for application daily-worship-test" application=daily-worship-test
level=info msg="Initializing [email protected]:<company>/gitops.git to /tmp/git-daily-worship-test32075355"
level=info msg="rm -rf /tmp/git-daily-worship-test32075355" dir= execID=6aead
level=info msg=Trace args="[rm -rf /tmp/git-daily-worship-test32075355]" dir= operation_name="exec rm" time_ms=0.7052700000000001
level=warning msg="temporarily disabling strict host key checking (i.e. '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'), please don't use in production"
level=info msg="git fetch origin --tags --force" dir=/tmp/git-daily-worship-test32075355 execID=006f2
level=error msg="Could not update application spec: could not find an image-name annotation for image daily-worship" application=daily-worship-test

Annotations:

annotations:
    argocd-image-updater.argoproj.io/image-list: <companyid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-name: image.name
    argocd-image-updater.argoproj.io/helm.image-tag: image.tag

Relevant values:

image:
  name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
  tag: develop-v0.20.0-33

[burnb]

So I did the things correctly, but it's still not updating the application:

level=debug msg="found 32 from 32 tags eligible for consideration" image="<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.20.0-33"
level=info msg="Setting new image to <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.21.0-181" alias= application=daily-worship-test image_name=daily-worship image_tag=develop-v0.20.0-33 registry=<accountid>.dkr.ecr.us-east-1.amazonaws.com
level=debug msg="target parameters: image-spec= image-name=image.name, image-tag=image.tag" application=daily-worship-test image=<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
time="2024-01-16T18:22:45Z" level=info msg="Successfully updated image '<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.20.0-33' to '<accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship:develop-v0.21.0-181', but pending spec update (dry run=false)" alias= application=daily-worship-test image_name=daily-worship image_tag=develop-v0.20.0-33 registry=<accountid>.dkr.ecr.us-east-1.amazonaws.com
level=debug msg="Using commit message: build: automatic update of daily-worship-test\n\nupdates image daily-worship tag 'develop-v0.20.0-33' to 'develop-v0.21.0-181'\n"
level=info msg="Committing 1 parameter update(s) for application daily-worship-test" application=daily-worship-test
level=info msg="Initializing [email protected]:<company>/gitops.git to /tmp/git-daily-worship-test32075355"
level=info msg="rm -rf /tmp/git-daily-worship-test32075355" dir= execID=6aead
level=info msg=Trace args="[rm -rf /tmp/git-daily-worship-test32075355]" dir= operation_name="exec rm" time_ms=0.7052700000000001
level=warning msg="temporarily disabling strict host key checking (i.e. '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'), please don't use in production"
level=info msg="git fetch origin --tags --force" dir=/tmp/git-daily-worship-test32075355 execID=006f2
level=error msg="Could not update application spec: could not find an image-name annotation for image daily-worship" application=daily-worship-test

Annotations:

annotations:
    argocd-image-updater.argoproj.io/image-list: <companyid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/helm.image-name: image.name
    argocd-image-updater.argoproj.io/helm.image-tag: image.tag

Relevant values:

image:
  name: <accountid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
  tag: develop-v0.20.0-33

in your case annotation should be

annotations:
    argocd-image-updater.argoproj.io/image-list: <companyid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/daily-worship.helm.image-name: image.name
    argocd-image-updater.argoproj.io/daily-worship.helm.image-tag: image.tag

[devopsidiot]

Thanks so much for responding: I did these things:

in your case annotation should be

annotations:
    argocd-image-updater.argoproj.io/image-list: <companyid>.dkr.ecr.us-east-1.amazonaws.com/daily-worship
    argocd-image-updater.argoproj.io/daily-worship.helm.image-name: image.name
    argocd-image-updater.argoproj.io/daily-worship.helm.image-tag: image.tag

And the only error that I got was time="2024-01-17T15:35:19Z" level=warning msg="Event could not be sent: Event \"daily-worship-test.17ab2c6a6ce998d6\" is invalid: involvedObject.namespace: Invalid value: \"argo\": does not match event.namespace" application=daily-worship-test

Since these argo applications are in the argo namespace and not the argocd namespace, am I just SoL for the time being?

[burnb]

It's just a warning that it can't send event to the k8s. And at this step it successfully sent git update.

[devopsidiot]

You literally made me cry with relief. Thank you so much for all your hard work. This needs to get adopted. This makes it work.

[strangnet]

I've integrated this PR with #672 since I needed both features, and initial tests shows that it's working. This should be a prioritised feature to be included in a release not too far away. Great work @burnb

[tomjohnburton]

@burnb you are a hero! Finally ArgoCD somewhat resembles the power of Flux

[jnovick]

Hey, I have been using this and it works great. I am cheering for this too. I can attest that it works when using aliases in the image list. It writes to a specified file as long as the directory of the file exists. The one piece that seemed to be missing is the ability to write to a directory that does not exist yet. I opened a separate PR for it since it does not directly relate to your PR, but if the maintainers prefer, they can combine our PRs.

[s-ledyakhov]

@burnb Could you take a minute and give me a hint too? It seems your version is what I need, but I get an error:

Could not update application spec: could not find an image-name annotation for image emm/arm

Annotations:

    argocd-image-updater.argoproj.io/image-list: arm=docker-registry.<name>/emm/arm
    argocd-image-updater.argoproj.io/arm.helm.image-name: deployment.arm.image.repository
    argocd-image-updater.argoproj.io/arm.helm.image-tag: deployment.arm.image.tag
    argocd-image-updater.argoproj.io/write-back-method: git
    argocd-image-updater.argoproj.io/write-back-target: "helmvalues:values_dev.yaml"

Values:

...
.
deployment:
  arm:
    image:
      repository: docker-registry.<name>/emm/arm
      tag: "9.0-1"

If I remove the write-back-target, then the .argocd-source-dev.yaml file looks like this, everything works fine

helm:
  parameters:
  - name: deployment.arm.image.repository
    value: docker-registry.<name>/emm/arm
    forcestring: true
  - name: deployment.arm.image.tag
    value: 9.0-57
    forcestring: true

If I add the write-back-target: "helmvalues:values_dev.yaml", then I get the error "could not find an image-name" I also tried write-back-target: "helmvalues" write-back-target: "helmvalues:../../values_dev.yaml" write-back-target: "helmvalues:/helm/emm-helm/values_dev.yaml"

Just in case, tree looks like this:

helm/
├── emm-helm
│   ├── values.yaml
│   ├── values_dev.yaml
│   ├── ...

I will be eternally grateful if you have time to help with this!

[burnb]

@s-ledyakhov I guess the problem is with parsing of your image link by this function:

func ParseNormalizedNamed(s string) (Named, error) {
	if ok := anchoredIdentifierRegexp.MatchString(s); ok {
		return nil, fmt.Errorf("invalid repository name (%s), cannot specify 64-byte hexadecimal strings", s)
	}
	domain, remainder := splitDockerDomain(s)
	var remote string
	if tagSep := strings.IndexRune(remainder, ':'); tagSep > -1 {
		remote = remainder[:tagSep]
	} else {
		remote = remainder
	}
	if strings.ToLower(remote) != remote {
		return nil, fmt.Errorf("invalid reference format: repository name (%s) must be lowercase", remote)
	}

	ref, err := Parse(domain + "/" + remainder)
	if err != nil {
		return nil, err
	}
	named, isNamed := ref.(Named)
	if !isNamed {
		return nil, fmt.Errorf("reference %s has no name", ref.String())
	}
	return named, nil
}

from the "github.com/distribution/distribution/v3/reference" package, coz this error doesn't go to the logs and in that case, it parses image a bit differently, so alias doesn't go anywhere, and this is the root of the problem - it operates with the image name.

[s-ledyakhov]

@s-ledyakhov I guess the problem is with parsing of your image link by this function:

from the "github.com/distribution/distribution/v3/reference" package, coz this error doesn't go to the logs and in that case, it parses image a bit differently, so alias doesn't go anywhere, and this is the root of the problem - it operates with the image name.

Thanks for the answer Do I understand correctly that this problem has been solved in your repository? I've tried installing your manifest, but I haven't been able to solve the problem yet Although my values-file look the same as the user's above

[burnb]

I think the problem is with your <name> in image. Maybe it is not in the lower case?

[s-ledyakhov]

I think the problem is with your <name> in image. Maybe it is not in the lower case?

No, I checked this point docker-registry.<companyname>.com where is one word written together in lowercase letters

[mubarak-j]

I can confirm this PR fixes the issues with nested image parameters, however, it does rewrite the value file to sort all yaml/helm keys alphabetically. was this introduced in this PR or in https://github.com/argoproj-labs/argocd-image-updater/pull/636? Can we avoid re-arranging helm parameters?

[burnb]

@mubarak-j fixed, but I'm not quite sure about why linter alerting, I can't reproduce it locally and all the messages not about the PR code.

[zagr0]

Hi, can the fix be moved forward to be merged? Thanks you very much in advance!

[mubarak-j]

@burnb looks like there were lint fixes merged recently, so perhaps rebasing with the main branch might fix these issues?

We had the build of this PR running in production for over a month now, and working as expected, thank you!

@pasha-codefresh @chengfang @jannfis Can you please review this PR?

[chengfang]

@burnb thanks for the fix. Can you rebase to pull in recent fixed in master, squash, and leave out changes that are not required to fix this issue? Dependency updates should be done via separate PRs.

[pasha-codefresh]

@mubarak-j could you please resolve conflicts ?

[mubarak-j]

@mubarak-j could you please resolve conflicts?

@burnb can you please do the honor? 🙏🏼

IMHO I think it's worth adding an example in the docs for using write-back-target with helmvalues, currently docs only cover kustomize use case. I could also do that in a separate PR.

[pasha-codefresh]

@burnb i also prefer resolve different issues in different PRs, can we split all the problems to different PRs? As first thing i am going to review this one https://github.com/argoproj-labs/argocd-image-updater/pull/651

[pasha-codefresh]

Left comment https://github.com/argoproj-labs/argocd-image-updater/pull/651#pullrequestreview-2080722475

[chengfang]

@burnb can you please take another look, rebase and resolve conflicts based on recent review comments? Thanks!

[pasha-codefresh]

@chengfang i think we should merge this one first https://github.com/argoproj-labs/argocd-image-updater/pull/725