Feature: additional actions/links to be renderd in sink Findings

[Dariusch]

trafficstars

Is your feature request related to a problem?

Most of our alerts are enriched with additional annotations like dashboard_url and runbook_url to make debugging and resolving issues faster. Not everything can be automated, sadly... Those annotations were previously used to render slack actions/links to the specified hyperlinks.

Describe the solution you'd like

Passing additional (external) links to the (slack_)sink configs which are then rendered when present on the alert.

Describe alternatives you've considered

Currently, only robusta-provided links to the UI are configured: https://github.com/robusta-dev/robusta/blob/master/src/robusta/integrations/slack/sender.py#L282-L303

Additional context

The example I am talking about is described here.

[arikalon1]

Thanks for reporting it @Dariusch

How would you configure those links? Does all alerts has those links? (dashboard_url, runbook_url) Would you want to configure it per alert? Or, it's something that converts given annotations to buttons, if those are available?

[Dariusch]

Not all of the alerts have those links.

Or, it's something that converts given annotations to buttons, if those are available? Yes

For example the PrometheusRules configured by the kube-prometheus-stack do not have those annotations.

A flux alert could link to a dashboard for example:

---
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: flux-rules
  namespace: monitoring
spec:
  groups:
    - name: flux
      rules:
        - alert: ReconciliationFailure
          expr: max(gotk_reconcile_condition{status="False",type="Ready"}) by (exported_namespace, name, kind) + on(exported_namespace, name, kind) (max(gotk_reconcile_condition{status="Deleted"}) by (exported_namespace, name, kind)) * 2 == 1
          for: 3m
          labels:
            severity: warning
            team: engineering
          annotations:
            summary: '{{ $labels.kind }} {{ $labels.exported_namespace }}/{{ $labels.name }} reconciliation failed'
            description: '{{ $labels.kind }} {{ $labels.exported_namespace }}/{{ $labels.name }} reconciliation failed for more than 10 minutes'
            dashboard_url: https://grafana.${domain}/d/flux-cluster/flux-cluster-stats?orgId=1&refresh=30s

Or add the runbook URL. Annotation and labels are a kv map.

I guess a proposed solution would be to add a list to the slack sink config which is mutated into the slack schema like

- slack_sink:
    # Other values
    actions:
      - type: button
        text: 'Runbook :green_book:'
        url: '{{ (index .Alerts 0).Annotations.runbook_url }}'

[aantn]

See also #735

[aantn]

FYI, we made some improvements around including links in Slack messages here: https://github.com/robusta-dev/robusta/pull/967

It doesn't handle existing links from annotations/labels on the alert. But we're thinking about expanding this to your use case, as well as showing links from annotations in the Robusta UI.