flagger icon indicating copy to clipboard operation
flagger copied to clipboard

Upgrading flagger from 1.40.0 to 1.41.1 causes the canaries to be rollout whithout changes in the deployment

Open ealogar opened this issue 8 months ago • 3 comments

Describe the bug

In kubernetes 1.32.3 when upgradding flagger version to 1.41.1 all the canaries in the cluster triggers a new rollout without any changes in the deployment that is being targetted.

{"level":"info","ts":"2025-04-15T17:17:58.828Z","caller":"controller/events.go:33","msg":"New revision detected! Scaling upcanary1","canary":"canary1"} {"level":"info","ts":"2025-04-15T17:17:58.830Z","caller":"controller/events.go:33","msg":"New revision detected! Scaling up canary2","canary":"canary2"} ... The same for all the 10 canaries I'm testing

To Reproduce

A k8s cluster in 1.32.3 and some canaries in it update flagger

Expected behavior

Without changes in the deployment no rollout should happen.

Additional context

  • Flagger version: 1.41.1
  • Kubernetes version: 1.32.3
  • Service Mesh provider:
  • Ingress provider: nginx

Similar to https://github.com/fluxcd/flagger/issues/1673

ealogar avatar Apr 15 '25 17:04 ealogar

I can reproduce it with kubernetes 1.31.5 as well

ealogar avatar Apr 15 '25 18:04 ealogar

Seeing this too with version 1.41.1 on Kubernetes version 1.31.4 in Amazon EKS

We have a suspicion it's being caused by .spec.service.headless https://github.com/fluxcd/flagger/blob/main/CHANGELOG.md#1410

ccystephenclinton avatar Apr 16 '25 10:04 ccystephenclinton

We're seeing the same issue. Still not sure what in Flagger causes that...

vkoshkarovroku avatar Jul 10 '25 21:07 vkoshkarovroku