prometheus-engine icon indicating copy to clipboard operation
prometheus-engine copied to clipboard

Support non-Endpoint-backed Services for AlertmanagerEndpoints

Open pintohutch opened this issue 2 years ago • 3 comments

The rule-evaluator is only able to route alerts to Alertmanagers backed by Kubernetes Endpoints.

However, valid cases exist to route to Alertmanagers outside of the current cluster.

See the discussion in https://github.com/GoogleCloudPlatform/prometheus-engine/discussions/576#discussioncomment-7128328 for context.

pintohutch avatar Oct 03 '23 14:10 pintohutch

Considering that Endpoints object is now deprecated, are there thoughts on the priority of this feature?

Ref: https://kubernetes.io/blog/2025/04/24/endpoints-deprecation/

We currently manually create Endpoint objects and point them towards our ILB. We get a bunch of deprecation notices which we've been consciously ignoring 😅.

From what I understand from the deprecation notice, we can still use Endpoints objects, it's just not recommended, so maybe not a big fire, but still would be nice if we could route to externalName services like on the discussion where this issue was generated from. We have exactly that same setup.

If it's of interest, our use case is as follows:

  • 1 core cluster where we deploy our main workloads
  • 3 regional specific clusters where we deploy client workloads (eu/asia/us)
  • We have Alertmanager running in our core cluster (eu) with managed rule evaluation in each of the clusters. We route the region rule evaluation to the core cluster via a LB.

b-n avatar Oct 22 '25 11:10 b-n

@lyanco or @bwplotka - any thoughts on this?

pintohutch avatar Nov 10 '25 15:11 pintohutch

Sounds like a solid feature to add.

bwplotka avatar Nov 13 '25 10:11 bwplotka