moira
moira copied to clipboard
moira-alert
Change color and state of metric when turn on the maintenance
Hello,
Is there any way to change color and state of metric when turning on the maintenance? For example, if we have error (red) state for some metric but we turn maintenance for it (because we wan't to ignore it for some time), as I understand we won't get any notification for it. But is it possible to change state from error to another?
Hi, Bulat! No, in this case Moira keeps metric's last state to show actual state change at the expiration of maintenance period. But, if you're using some automation to control moira triggers (ex.: https://github.com/moira-alert/moira-trigger-role or api client) you can use trigger expression parameter instead of (warn/error limits)&maintenance.
trigger with params WARN >= 1; ERROR >= 10 will be checked equally as
trigger with expression "t1 >= 10 ? ERROR : (t1 >= 1 ? WARN : OK)"
For example:
You have trigger with expression "t1 >= 10 ? ERROR : (t1 >= 1 ? WARN : OK)" and you want to do something so many unwanted alerts can be produced, but you need the trigger to be "OK". So you can simply change current expression to "OK". But don't forget to set required trigger state in case when Moira has no metrics received (ttl_state). Otherwise you'll be notified in case of NODATA.
Thank you, @kamaev !
In this case, is it possible to make this way:
- we turn maintenance for some error (red) metric on
- this metric doesn't change its color and state in "Current state" area, but in this area:
we will have this view:
I mean just to make another view for counts of error/ok/warn area - to add for example "blue" counter for maintenance.
If this is not possible, could we make a month period request? Or is there another soultion to solve it?
Talked to Arkady, we decided to keep this issue open (sorry, Sasha!) 2.3 release is coming soon so we unfortunately have finite number of tasks to close. But we will definitely add the monthly maintenance option in future releases. Task to show colorized counter of metrics with maintenance turned on takes more discussion since we have some issues with its realization.
@borovskyav раз добавил в 2.5, опиши задачу, что изменится и как будет?
Давайте на летучке в понедельник вместе подумаем, у меня есть несколько решений, но я не могу выбрать между сортами)
на летучке в понедельник вместе подумаем
@borovskyav запиши себе на мартовское планирование, плез
