tofu-controller icon indicating copy to clipboard operation
tofu-controller copied to clipboard

Incorrect gotk_reconcile_condition metric after migration to v0.16.0-rc.3

Open renato0307 opened this issue 2 years ago • 8 comments
trafficstars

Hi,

I've upgraded from v0.14.0 to v0.16.0-rc.3.

The gotk_reconcile_condition for the Ready condition is not being correctly reported.

  • Terraforms are Ready
  • Metric reports Unknown=1 and True/False=0

Do you have any idea the cause? Could you please help me troubleshoot this issue?

Thank you in advance.

More details:

v0.16.0-rc.3

image

image

image

v0.14.0

image

image

renato0307 avatar Oct 26 '23 17:10 renato0307

Ran the controller with log-level=trace:

image

renato0307 avatar Oct 26 '23 18:10 renato0307

Thank you @renato0307 for reporting this.

Maybe it's because we have recently changed the code base to match that of Flux v2 GA.

@luizbafilho @yitsushi could you help take a look please?

cc @LappleApple

chanwit avatar Oct 27 '23 05:10 chanwit

Anyone working on this @LappleApple @chanwit? May I give it a go?

renato0307 avatar Nov 09 '23 13:11 renato0307

Hey @renato0307, all yours. :) I'll assign it to you.

lasomethingsomething avatar Nov 09 '23 13:11 lasomethingsomething

@renato0307 : Go for it, if any of us picked it up, we assign ourselves.

yitsushi avatar Nov 09 '23 13:11 yitsushi

Looks like the problem is on the defer.

The defer uses the original terraform var which contains the Unknown condition.

It is overwriting the metric written here, using the reconciledTerraform var.

renato0307 avatar Nov 09 '23 18:11 renato0307

Hi team, please evaluate this fix.

adonispd avatar Mar 25 '24 14:03 adonispd

Thank you for bringing this up @adonispd could anyone please help use clarify this bug again? Happy to help review and land a fix if someone could point things out clearly.

chanwit avatar Mar 25 '24 15:03 chanwit

Hi @chanwit, I could reproduce this issue in the v0.16.0-rc.4. Wrote a short summary on the PR. #1287

TarasLykhenko avatar Apr 26 '24 15:04 TarasLykhenko

@chanwit any idea when the team will create a new release candidate?

ricardomiguel-os avatar May 06 '24 15:05 ricardomiguel-os