cortex icon indicating copy to clipboard operation
cortex copied to clipboard

AlertManager CRD

Open mjallday opened this issue 5 years ago • 11 comments

Please let me know if there's an existing issue, I did a search but didn't see anything related to this yet.

I'm a big fan of the PrometheusRule CRD and am looking for something similar to configure AlertManager to allow declaring rules for alerting via a CRD.

  • https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/user-guides/alerting.md
  • https://github.com/prometheus-operator/prometheus-operator/search?q=PrometheusRule

I'm not sure of the state of the Cortex Project yet but am curious if there has been any development on this yet or if not, where would be a suitable place for this logic to live?

mjallday avatar Mar 23 '21 03:03 mjallday

This looks like something to build on top of Cortex (eg. automation to generate alertmanager config based on CRD) and not something to do within Cortex. I think supporting Kubernetes CRDs is out of the scope of Cortex.

pracucci avatar Mar 23 '21 08:03 pracucci

i see the other gh repo on the cortex org is a helm chart which is kubernetes specific. is there interest in doing something similar for k8s CRDs? or making it part of the helm chart?

i'm not super familiar with how the project sees these kinds of additions so looking for advice. we might be able to assist in development since we want a gitops driven deployment for cortex alerts

mjallday avatar Mar 23 '21 19:03 mjallday

We have the next Cortex community call scheduled for 1st April. If it's not too late, I would suggest you to join it and discuss it in that occasion, otherwise I can try to get the attention of other Cortex maintainers on this, since I don't have a quick answer / opinion.

pracucci avatar Mar 24 '21 09:03 pracucci

For what it's worth we are in the same boat as you and decided to not use Cortex's Alertmanager altogether and use the one from the Prometheus Operator so we could use the CRDs. It's not ideal and I would like to use the Cortex one, but it just doesn't suit our needs.

jargelo avatar Mar 24 '21 10:03 jargelo

Thanks, are these the relevant details for the Cortext community?

  • https://docs.google.com/document/d/1shtXSAqp3t7fiC-9uZcKkq3mgwsItAJlH6YW6x1joZo/edit#
  • https://groups.google.com/g/cortex-monitoring/search?q=community

i can't seem to see an event scheduled for 1st april. this google group seems a little dead so i'm guessing there's somewhere better to see this @pracucci

mjallday avatar Mar 26 '21 19:03 mjallday

Thanks, are these the relevant details for the Cortext community?

@mjallday Yes!

pracucci avatar Mar 29 '21 07:03 pracucci

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jun 27 '21 12:06 stale[bot]

We're also looking for similar functionality and this is the closest thing I could find https://github.com/bolindalabs/cortex-alert-operator/

william-tran avatar Dec 09 '21 15:12 william-tran

Cool, we just open sourced our version https://github.com/verygood-ops/open-metrics-multi-tenancy-kit

open-metrics-multi-tenancy-kit implements multi-tenancy for metrics collection and rules management in Cortex environments, configured via OpenMetricsRule Kubernetes resource.

mjallday avatar Dec 13 '21 17:12 mjallday

will leave mine here as well https://github.com/slamdev/cortex-rule-watcher

slamdev avatar Apr 24 '22 10:04 slamdev

I think this a good Idea we should revisit

friedrichg avatar Jun 12 '25 21:06 friedrichg