Harbor webhook issues with fluxv2

[venkatesh-mb]

Our flux auto deployment woroking without any issues.

But we are having loot of inconscity with Webhook to deploy latest image as soon as image pushed. As soon as we configure 1st time it's working without any issues, later after coupole tages it's stopped working. We as below template to test our webhook.

apiVersion: notification.toolkit.fluxcd.io/v1beta1
kind: Receiver
metadata:
  name: harbor-receiver
  namespace: tenant-dev
spec:
  type: harbor
  secretRef:
    name: webhook-token
  resources:
    - apiVersion: image.toolkit.fluxcd.io/v1beta1
      kind: ImageRepository
      name: application-testing

Any idea how can we troubleshoot this issue?

and second question relevent to same topic.

We have around 50 applications in each environment. Do we have create 50 yamls files for every imageRepositry? or is there any way we can manage all at a time.

[stefanprodan]

Any idea how can we troubleshoot this issue?

Please look at the notification-controller logs and see if the receiver runs into errors, if there are no issues there, look at the image-reflector-controller logs. Also the status conditions of the ImageRepository object could tell you if it runs into an error or not. The recevier does the same thing as flux reconcile image repository <name> -n <namespace>, if you run the command does it pull the latest tags or not?

Do we have create 50 yamls files for every imageRepositry? or is there any way we can manage all at a time.

There is no way to register images bulk, you could use some script to generate all the YAMLs from a list of images.

[venkatesh-mb]

There are no errors with image-reflector-controller, notification-controller, and ImageRepository shows last run time according to the interval time specified at ImageRepository and it's working fine.

but we want to run this with webhook and it didn't deployed with as soon as image pushed to harbor.

flux reconcile image repository <name> -n <namespace>, this command pull the latest tags.

[stefanprodan]

My guess is that the image is not really there in the Harbor API when you trigger the receiver, what happens if you add let's say a 10 seconds delay before you call the Flux receiver?

[venkatesh-mb]

that would be fine, but where we could configure this delay time?

[venkatesh-mb]

We believe this is something which we need to add at receiver level but syntax not allowing us to add.

[stefanprodan]

I suspect this is a Harbor bug where the webhook is triggered by the query doesn't return new tags. Switch to a generic receiver, then in the CI job that pushes images to Harbor call the Flux webhook after docker pull works for that tag.