sloth
sloth copied to clipboard
SLOs in Prometheus
I have deployed latest slot version(v0.11.0) with Helm.
i have defined a PrometheusServiceLevel :
kind: PrometheusServiceLevel
metadata:
name: sloth-example-profile
namespace: monitoring
labels:
prometheus: prometheus
role: alert-rules
app: sloth
spec:
service: example-profile
slos:
- alerting:
name: example_profile_availability_alert
pageAlert:
labels:
severity: warning
ticketAlert:
labels:
severity: info
name: example_profile_availability
objective: 99
sli:
events:
errorQuery: count_over_time((sum(up{namespace="exampleplatform",pod =~ "example-profile.*"})<1)[{{.window}}:])
totalQuery: count_over_time(sum(up{namespace="exampleplatform",pod =~ "example-profile.*"})[{{.window}}:])
the PrometheusRule is successfully generated:
NAME SERVICE DESIRED SLOS READY SLOS GEN OK GEN AGE AGE
sloth-example-profile example-profile 1 1 true 3m19s 99m
but the Prometheus is not finding metrics for 'slo':
And without this the Grafana dashboards are just empty.
Am i missing something? Is there any additional step for this?
Maybe prometheus does not ingest PrometheusRules because either label or namespace selectors for rules are out of scope.
I am having the same problem with 0.11. @d2k-klin did you manage to solve the issue?
@alexanderjardim Probably you're missing the label as @klubi said.
Look in your prometheus config for ruleSelector
to confirm the label you're using.