logging-operator
logging-operator copied to clipboard
kube-logging
Make severity configurable in PrometheusRules
Is your feature request related to a problem? Please describe. Currently the severity is hardcoded in the PrometheusRules like here
Describe the solution you'd like It would be great to have them configurable via the Logging CR e.g.:
spec:
fluentd:
metrics:
prometheusRules: true
severity: my-custom-severity
fluentbit:
metrics:
prometheusRules: true
severity: my-custom-severity
syslogNG:
metrics:
prometheusRules: true
severity: my-custom-severity
Describe alternatives you've considered My current workaround is to disable the PrometheusRules and deploy them in the way I want
Additional context Logging Operator docs: https://kube-logging.dev/docs/operation/alerting/
Good idea, but I don't have any free cycles currently. We are looking for contributors to take on "good first issues" like this!
@aditya7302 that is generated from the go code. I recommend to check the Metrics type that could be extended with a severity field and applied everywhere where a prometheusrule is defined, for example here
By looking at the code, my problem is, that using this method we would have to use the same severity for all the prometheusrules that are defined for a single metrics endpoint. For example in case of fluentbit there are two rules defined, one with warning and one with critical severity. How would the proposed severity setting work in this case? 🤔
@pepov @aditya7302 maybe my comment now is a bit controversial to remove the PrometheusRules completely or making the whole Rules block configurable.
I would add a list based override option that would allow setting overrides for the rules based on the list index. Also we could define a type that would allow overriding other fields as well for the rule.
For example the below metrics config would keep the first rule unmodified but would change a few properties of the second rule.
metrics:
prometheusRules: true
prometheusRuleOverrides:
- {}
- for: 1m
labels:
severity: warning
additionalLabel: b
I think we could also allow a list item to be removed if we set the corresponding item to nil.
@pepov I have made some changes to make severity configurable. Please review the changes.