SLO dependencies / levels / hierarchy

[prein]

Hi, would implementing SLO relations be an idea worth considering? Where I think would it be useful:

• alerting - mute dependent / children SLO alerts if parent SLO is firing
• visualization - in a system with 10s or 100s of SLOs it would allow for creating less cluttered top level lists from where one would drill down to deeper levels, could also help select SLOs for public status page
• visualization - hierarchical tree view
• alerting - setting alert severity based on SLO level

[metalmatze]

Generally speaking, Pyrra follows Prometheus' approach using labels instead of using hierarchy. Do you think you can model the same using various labels?

If not, examples would be great. Thanks!

[prein]

Thanks, I'll look into what I can achieve with labels alone. However there will still be the UI part. I like how Pyrra's UI is clean. It gets cluttered though with 10s of services with 10+ of SLOs each. Some form of grouping / filtering would be useful for me.

Also - something to consider as inspiration from Prometheus could be alert inhibition.

[warmfusion]

I've been thinking about this problem for a while. Defining SLO against services makes sense (eg http request success) but i'd love to create a more product aligned SLO that can group services together.

As a Senior Developer When I provide a service to my customer through multiple services I want to have a Customer aligned SLO for my overall product offering that relates to the SLO we've set on our individual services So that I can quickly understand our overall availability

One idea that came to mind was simply defining SLO's against the Pyrra generated field terms, and then using labels to indicate some sort of 'grouping' attribute as suggested above.

Further to @prein 's thoughts, it'd be great if there was a way of indicating 'parent/child' relationships in some capacity such that if an SLO is a compound of other SLO it can be displayed elegantly (or even just have links to eachother)

Its not a simple problem to solve, and may be sufficient to give an example of a compound SLO referencing Pyrra metrics?