How to turn off `on_kubernetes_any_resource_all_changes` trigger that sends messages to telegram in every pod creation.

[IdeoG]

trafficstars

subj, i found it was declared in ConfigLoader as internal trigger.

Notifications on every pod creation/deletion is redudant in our logic, or we just want to separate telegram alerts for release notifications and alerts. it would be helpful to do that.

[arikalon1]

Thank you for reporting this @IdeoG

Do you have any customPlaybooks configured? If so, can you please share your customPlaybooks section ? (Please make sure not to share any sensitive value)

[IdeoG]

builtinPlaybooks:
- triggers:
    - on_kubernetes_warning_event:
        namespace_prefix: jupyterhub
  actions:
    - resource_babysitter: {}

so, it's a part of my values.yaml config file.

[arikalon1]

Typically, Robusta installation has the values.yaml file, which is the defaults file, and the generated_values.yaml file, which is the overrides file.

Is it configured this way on your installation as well?

Regardless, you're welcome to join our Slack community. We're available there as well. You can find the link to our Slack channel here: https://home.robusta.dev/

[IdeoG]

Oh, i see, suddenly, i used trainers.values.yaml filename, so i renamed it to values.yaml and hope it will help. Thank you!

[IdeoG]

Hmm, it didn't work. Pod created/deleted events are coming to my tg group. Is there any ideas

[arikalon1]

Can you please share your generated_values.yaml file (without the sensitive values, if there are any)

[IdeoG]

builtinPlaybooks:
- triggers:
    - on_kubernetes_warning_event:
        namespace_prefix: jupyterhub
  actions:
    - resource_babysitter: {}


globalConfig:
  signing_key: ...
  account_id: 631b58c8-d88d-4c63-b7c8-94e0be313823
sinksConfig:
- telegram_sink:
    name: personal_telegram sink
    bot_token: ...
    chat_id: -1001758586891
disableCloudRouting: true
runner:
  sendAdditionalTelemetry: false

[arikalon1]

Can you explain when you'd like to get a notification on telegram? For example, when a deployment is created, or when a statefulset is changed?

[IdeoG]

Sure! My use cases:

1. Deployment update was started and finished
2. Deployment was created and creation was successful
3. Any k8s object warnings from namespace
4. Pod errors, such as OomKiller, Loopbackoff

Thank you for your help!

[arikalon1]

Thank you

In general, when adding new playbooks, you should use the customPlaybooks value, and not the builtinPlaybooks

Specifically, regarding your use cases:

1. Deployment update

- triggers:
  - on_deployment_update: {}
  actions:
  - resource_babysitter: 
      fields_to_monitor:
      - spec

This will send a change event, whenever the deployment spec change.

2. Deployment creation

- triggers:
  - on_deployment_create: {}
  actions:
  - create_finding:
      aggregation_key: "Deployment Created"
      severity: INFO
      title: "New deployment created"
      description: "New deployment: name - $name on namespace - $namespace"

3. k8s warnings from namespace This might be too spammy, but if you do want it, this is the playbook you need:

- triggers:
  - on_kubernetes_warning_event_create:
      exclude: ["NodeSysctlChange"]
  actions:
  - event_report: {}
  - event_resource_events: {}

4. Pod crash loops, oom kills etc are configured by default. You don't need to modify anything

So, your generated_values.yaml file should have:

customPlaybooks:
- triggers:
  - on_deployment_update: {}
  actions:
  - resource_babysitter: 
      fields_to_monitor:
      - spec

- triggers:
  - on_deployment_create: {}
  actions:
  - create_finding:
      aggregation_key: "Deployment Created"
      severity: INFO
      title: "New deployment created"
      description: "New deployment: name - $name on namespace - $namespace"

- triggers:
  - on_kubernetes_warning_event_create:
      exclude: ["NodeSysctlChange"]
  actions:
  - event_report: {}
  - event_resource_events: {}

Please let us know if you have additional questions

[arikalon1]

@IdeoG , did the above configuration solve your use cases?

[aantn]

Closing due to inactivity. @IdeoG please feel free to re-open if you have issues.