feat: implement webhook receiver for Docker, GHCR and Harbor to receive triggers for image update

[binhnguyenduc]

trafficstars

This PR is to address https://github.com/argoproj-labs/argocd-image-updater/issues/1 and is essentially a continuation of https://github.com/argoproj-labs/argocd-image-updater/pull/284

This pull request introduces a webhook server to handle container registry events, enabling automated image updates in ArgoCD applications. Key changes include adding webhook server functionality, integrating registry-specific webhook handlers, and providing configuration options for the webhook server.

Webhook Server Implementation:

• Added webhook server functionality: Implemented a new WebhookServer class to handle registry events, including initialization, event handling, and graceful shutdown. (cmd/main.go, cmd/run.go, pkg/webhook/docker.go, pkg/webhook/docker_test.go) [1] [2] [3] [4] [5] [6]
• Introduced WebhookOptions and CLI command: Added a new webhook CLI command with configuration options such as port, registry secrets, and ArgoCD integration settings. (cmd/webhook.go)

Registry-Specific Webhook Handlers:

• Docker Hub webhook handler: Created a handler for Docker Hub registry events, including payload validation using HMAC-SHA256 signatures and event parsing. (pkg/webhook/docker.go, pkg/webhook/docker_test.go) [1] [2]

Configuration Enhancements:

• Updater configuration: Introduced UpdaterConfig to encapsulate settings for image updates, such as Git commit details and concurrency limits. (pkg/argocd/updater_config.go)

Integration with ArgoCD:

• ArgoCD client initialization: Added logic to initialize ArgoCD clients based on the application API kind (kubernetes or argocd) for webhook server operations. (cmd/run.go, cmd/webhook.go) [1] [2]

These changes collectively enhance the automation capabilities of the ArgoCD Image Updater by enabling it to respond to registry events and update applications accordingly.

I've also made sure to address concerns from the original PR:

• the feature is guarded behind a flag enable-webhook and is off by default
• user can set their own port via webhook-port

I have not update the manifests to add new Service and Ingress. If needed, let me know.

Disclaimer: the integration with Harbor is well tested and currently running on our own cluster. Unfortunately, I don't have access to Dockerhub and GHCR to test thoroughly, so any help is welcomed.

[codecov-commenter]

Codecov Report

:x: Patch coverage is 51.41956% with 308 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 60.80%. Comparing base (2cd8c7d) to head (5ac6081). :warning: Report is 16 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/webhook/server.go	0.00%	141 Missing :warning:
cmd/webhook.go	27.65%	67 Missing and 1 partial :warning:
cmd/run.go	3.22%	60 Missing :warning:
pkg/webhook/harbor.go	89.83%	9 Missing and 3 partials :warning:
pkg/argocd/updater_config.go	0.00%	11 Missing :warning:
pkg/webhook/docker.go	91.78%	4 Missing and 2 partials :warning:
pkg/webhook/ghcr.go	94.17%	4 Missing and 2 partials :warning:
pkg/webhook/webhook.go	87.09%	3 Missing and 1 partial :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1159      +/-   ##
==========================================
- Coverage   63.27%   60.80%   -2.48%     
==========================================
  Files          15       22       +7     
  Lines        2358     2990     +632     
==========================================
+ Hits         1492     1818     +326     
- Misses        771     1068     +297     
- Partials       95      104       +9     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[chengfang]

UpdaterConfig in pkg/argocd/updater_config.go: is it possible to use other existing struct, instead of creating a new one?

webhook secrets: can we keep them in a secret file, similar to https://argo-cd.readthedocs.io/en/latest/operator-manual/webhook/#2-configure-argo-cd-with-the-webhook-secret-optional ?

[chengfang]

I have not update the manifests to add new Service and Ingress. If needed, let me know.

Yes, the service, ingress and related manifests would be great to have.

[chengfang]

How do you currently use this feature, using the standalone run command with webhook enabled, or using the standalone webhook command, or run image updater workload in cluster with webhook enabled?

[binhnguyenduc]

I have not update the manifests to add new Service and Ingress. If needed, let me know.

Yes, the service, ingress and related manifests would be great to have.

I'll update as soon as I can

[binhnguyenduc]

How do you currently use this feature, using the standalone run command with webhook enabled, or using the standalone webhook command, or run image updater workload in cluster with webhook enabled?

Currently, we enable webhook with the run command (via env var) and reduce the interval to 30m or 60m. That way, we have both the webhook trigger and registry scanner running side by side. One way is to potentially run the webhook command, with the registry scanner off altogether, but we haven't actually done that in production.

[binhnguyenduc]

UpdaterConfig in pkg/argocd/updater_config.go: is it possible to use other existing struct, instead of creating a new one?

webhook secrets: can we keep them in a secret file, similar to https://argo-cd.readthedocs.io/en/latest/operator-manual/webhook/#2-configure-argo-cd-with-the-webhook-secret-optional ?

I'll see what I can do.

[chengfang]

Will merge this as an experimental feature and will continue to improve it.

[phil-monroe]

@binhnguyenduc + @chengfang - This PR is incredibly well-timed for what I think is an issue we're encountering at my job, with one caveat: we're using Artifactory. If we were to contribute a PR that followed the patterns of this PR and provided an Artifactory handler, would you want to include it with these current handlers as well?

[chengfang]

@phil-monroe PRs are welcome to enhance the webhook feature!

[ktzsolt]

I am trying to setup the master version of the repo to use the webhook with harbor, but when I set the webhook.harbor-secret in argocd-image-updater-secret I get this: level=error msg="Failed to process webhook: invalid webhook signature"

If I don't set the webhook.harbor-secret or set it to empty string then I get this: level=error msg="Failed to process webhook: missing webhook signature"

Please help how should I set the signature. I was trying to use a random passphrase.

[ktzsolt]

I am trying to setup the master version of the repo to use the webhook with harbor, but when I set the webhook.harbor-secret in argocd-image-updater-secret I get this: level=error msg="Failed to process webhook: invalid webhook signature"

If I don't set the webhook.harbor-secret or set it to empty string then I get this: level=error msg="Failed to process webhook: missing webhook signature"

Please help how should I set the signature. I was trying to use a random passphrase.

What I did to make it work is that I deleted with kustomize patch the argocd-image-updater-secret so it works now without authentication