[kube-prometheus-stack] alertmanager inhibit rules not working

[dioguerra]

Describe the bug a clear and concise description of what the bug is.

alertmanager.config.inhibit_rules not working

What's your helm version?

v3.8.0

What's your kubectl version?

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:38:50Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.3", GitCommit:"c92036820499fedefec0f847e2054d824aea6cd1", GitTreeState:"clean", BuildDate:"2021-10-27T18:35:25Z", GoVersion:"go1.16.9", Compiler:"gc", Platform:"linux/amd64"}

Which chart?

kube-prometheus-stack

What's the chart version?

36.2.0

What happened?

The inhibit rules for the alertmanager are not working.

In fact, i did a quick search in templates and there is nothing that would create alertmanagerConfig object.

What you expected to happen?

alertmanagerConfig object to be created with the required configs to silence alert

How to reproduce it?

alertmanager:
  config:
    inhibit_rules:
    - source_matches:
      - 'alertname = CPUThrottlingHigh'
      - 'container = node-exporter'
    - target_matches:
      - 'alertname = CPUThrottlingHigh'
      - 'container = node-exporter'  

Enter the changed values of values.yaml?

alertmanager: config: inhibit_rules:

Enter the command that you execute and failing/misfunctioning.

Command does not fail, but instead no manifest is created for the inhibit rule. If i create the manifest manually, the alerts are not created. I was actually expecting a silence rule to be created in the alertmanager server, but the alerts just disapear (i suspect they are neglected at ingest time?) Still not sure

This actually matches the crd specification for the alertmanagerConfig

apiVersion: monitoring.coreos.com/v1alpha1
kind: AlertmanagerConfig
metadata:
  name: silence-node-exporter
spec:
  inhibitRules:
  - sourceMatch:
    - name: alertname
      regex: false
      value: CPUThrottlingHigh
    - name: container
      regex: false
      value: node-exporter
  - targetMatch:
    - name: alertname
      regex: false
      value: CPUThrottlingHigh
    - name: container
      regex: false
      value: node-exporter

Anything else we need to know?

No response

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[mossad-zika]

This InfoInhibitor alert just spams random messages to me ...

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale[bot]]

This issue is being automatically closed due to inactivity.

[stale[bot]]

This issue is being automatically closed due to inactivity.