Grafana dashboard not generates metrics to Burning rate

[Eddie4Frost]

Hello guys, I hope you are doing great! Can you please help me, where am I going to not generate data from burning rate and Budget? Thank you in advance!

[slok]

Hi @Eddie4Frost!

In order to help you, I need more context :D

• Sloth logs.
• SLO.
• Are the metrics on the prometheus.
• The generated rules are loaded on Prometheus.
• ...

[Eddie4Frost]

Hi @slok, my bad! Thank you for your response, below: The metrics available on Prometheus: The sloth log: INFO[0000] SLI plugins loaded plugins=0 svc=storage.FileSLIPlugin version=v0.9.0 window=30d INFO[0000] SLO period windows loaded svc=alert.WindowsRepo version=v0.9.0 window=30d windows=2 INFO[0000] Loading Kubernetes configuration... version=v0.9.0 INFO[0000] Kubernetes controller running version=v0.9.0 window=30d INFO[0000] starting controller controller-id=sloth lib=kooper service=kooper.controller version=v0.9.0 window=30d INFO[0000] Hot-reload http server listening addr=":8082" version=v0.9.0 window=30d INFO[0000] Metrics http server listening addr=":8081" version=v0.9.0 window=30d INFO[0000] OS signals listener started version=v0.9.0 window=30d INFO[0000] Hot-reload manager running version=v0.9.0 window=30d INFO[0001] Hot-reload triggered from http webhook version=v0.9.0 window=30d INFO[0001] SLI plugins loaded plugins=15 svc=storage.FileSLIPlugin version=v0.9.0 window=30d

The rule:

Grafana Query Inspector query: Object request:Object url:"api/datasources/proxy/1/api/v1/query_range" method:"POST" data:Object query:"1-( sum_over_time( ( slo:sli_error:ratio_rate1h{sloth_service="myservice",sloth_slo="requests-availability"} * on() group_left() ( month() == bool vector(12) ) )[32d:1h] ) / on(sloth_id) ( slo:error_budget:ratio{sloth_service="myservice",sloth_slo="requests-availability"} *on() group_left() (24 * days_in_month()) ) )" start:1638327600 end:1641006000 step:3600 hideFromInspector:false response:Object status:"**success**" data:Object resultType:"matrix" result:Array[0]

Thank you

[slok]

Do other SLOs work for you, or happen on all of them?. Maye it's just that you aren't burning anything (rare, but who knows...) so there's nothing to chart. To test, you can use the "fake" plugin to simulate different burn rates (this should show you something always):

---
apiVersion: sloth.slok.dev/v1
kind: PrometheusServiceLevel
metadata:
  name: slo-test
  labels:
    prometheus: prometheus
    role: alert-rules
spec:
  service: "test"
  slos:
    - name: "five-nines"
      objective: 99.999
      sli:
        plugin:
          id: "sloth-common/fake"
          options:
            burn_rate: "0.65"
            jitter_percent: "10"
      alerting:
        name: "TestSLOFiveNinesFailing"
        pageAlert:
          disable: true
        ticketAlert:
          disable: true

[mccricardo]

@slok do you think adding a vector(0) in this case would make sense?

1-(
  sum_over_time(
    (
       slo:sli_error:ratio_rate1h{sloth_service="${service}",sloth_slo="${slo}"}
       * on() group_left() (
         month() == bool vector(${__to:date:M})
       )
    )[32d:1h]
  )
  / on(sloth_id)
  (
    slo:error_budget:ratio{sloth_service="${service}",sloth_slo="${slo}"} *on() group_left() (24 * days_in_month())
  ) or vector(0)
) 

[mo3athi]

Hi @slok I do have the same problem, here's a screenshot for the fake plugin as you mentioned on your last comment:

I'm using the helm chart version I don't see anything wrong on the sloth pod logs: INFO[0000] SLI plugins loaded plugins=0 svc=storage.FileSLIPlugin version=v0.9.0 window=30d INFO[0000] SLO period windows loaded svc=alert.WindowsRepo version=v0.9.0 window=30d windows=2 INFO[0000] Loading Kubernetes configuration... version=v0.9.0 INFO[0000] Kubernetes controller running version=v0.9.0 window=30d INFO[0000] Hot-reload http server listening addr=":8082" version=v0.9.0 window=30d INFO[0000] Metrics http server listening addr=":8081" version=v0.9.0 window=30d INFO[0000] Hot-reload manager running version=v0.9.0 window=30d INFO[0000] OS signals listener started version=v0.9.0 window=30d INFO[0000] starting controller controller-id=sloth lib=kooper service=kooper.controller version=v0.9.0 window=30d INFO[0001] Hot-reload triggered from http webhook version=v0.9.0 window=30d INFO[0001] SLI plugins loaded plugins=15 svc=storage.FileSLIPlugin version=v0.9.0 window=30d Thank you!