Feature Request : Support Amazon Managed Service for Prometheus with Sigv4 Authentication

[hermes-pimentel]

Currently, it is not possible to use Prometheus managed by AWS (AMP) as a URI endpoint to search for metrics. Many companies are migrating to this solution. Currently my environment uses a Prometheus with remote_write to send the metrics collected from JMX to AMP, but it is not possible to use CruiseControl. Having AMP support with authentication via SigV4 from AWS would ease the process of reading metrics and remove the burden of maintaining prometheus for large and complex environments.

[wyuka]

This sounds interesting. I think it would definitely add value. On the other hand, I am not entirely convinced that the code required to do authentication with a third-party service should be in the Cruise Control core code. One possible solution I can think of is doing it as a plug-in, but I am open to discussion on this.

Would you be willing to contribute this feature @hermes-pimentel ?

[igorleoem]

It would be really nice to have this feature running in Cruise Control. AMP is a really interesting solution that enables Prometheus in a few minutes and leaves us without worries about the infrastructure. To have this would be a most!

[mohitpali]

@wyuka I agree that this should be a Plugin. An Interface PrometheusAuthProvider can be implemented and then users can use their own AWSSigV4AuthProvider.

If Cruise Control maintainers are open to add some AWS SDK dependencies, then we can provide a non-default implementation as well for ease of use.

[pablomparada]

Hello,

I would like to implement this feature with some guidance tough. As far I can understand this may need AWS dependencies and thus shouldn't be added to core. Should this implementation fall in cruise-control-metrics-reporter module then?

If it does, is there any design you would like to pursue?

[hermes-pimentel]

Hi @wyuka and @mohitpali, I'm willing to help to implement this feature with @pablomparada, however we need some guidance in relation to the issues raised. Where can this feature be implemented? Should this implementation fall in cruise-control-metrics-reporter module as suggested above?

[maiconbaum]

Come on guys, this is a really nice feature - I should say it's a must-have feature in today's world, but this is just my opinion). @hermes-pimentel and @pablomparada will save lives with this improvement.. So in God's name can you answer their questions? :(

[mohitpali]

@wyuka since you have more context on the code placement here, could you please help derive the approach pleas.

The cruise-control-metrics-reporter package though contains the module that is copied on Kafka brokers to assist with storing raw metrics. The Prometheus way does not need the cruise-control-metrics-reporter module.

[KovtunV]

Hi guys, any progress? :)

[hermes-pimentel]

Hi @wyuka, can we add this implementation in cruise-control-metrics-reporter module?