Support a prometheus / openmetrics endpoint

[jespersoderlund]

To be a part of an openmetrics centric metrics infrastructure it would be useful if the ratelimit service natively supported a prometheus metrics endpoint without having to go past a statsd bridge

[ysawa0]

Great idea, that makes sense to me.

[irl-segfault]

+1, would be super useful! Is there any kind of shim or adapter people can use in the meantime?

[irl-segfault]

^ answer: use statsd-prom exporter

[ethernoy]

I think I can work on adding Prometheus instrumentation in ratelimit, but I believe we should work out a list of possible metric to be implemented first.

[zirain]

this PR tried to fix this by adding prometheus_sink, but it won't. a valid metric in stats may be invalid in prometheus, e. g. ratelimit.service.rate_limit.httpbin.header_match_\get.near_limit

[ramaraochavali]

@zirain Do you know what needs to be done to fix ^^?

[upgle]

Hi, I've abstracted an existing structure that was fully coupled to gostats and applied Prometheus metrics to it. I would like to contribute this code. (not using gostats prometheus_sink)

I tried to contribute a small change to this project last time, but no one was interested and it closed automatically. I was wondering if there are other procedures like mailing.

Thank you

Label based metric for prometheus

[jespersoderlund]

I've contributed a few things over the last couple of years and the maintainers have picked-it up in reasonable time.

Sorry, to hear that your previous PR was closed, hope that your contributions can get reviewed this time.

Not a maintainer myself, so I can't directly help-out.

I'm keen to see this feature as you probably can guess since I created the issue :)

[jespersoderlund]

@upgle One thing that the current setup going through statsd => prometheus allows us to do is to not publish all metrics since some it can be quite high cardinality.

So we only publish the "near_limit" and "over_limit" metrics.

Will your implementation allow something similar, like having an allow-list of metrics to publish?

[upgle]

Thank you for your kind reply. I understand your concerns about cardinality.

I checked this out, other metrics other than "near_limit" and "over_limit" are all useful metrics (e.g. redis, memcached, localcache...), and these metrics do not have high cardinality.

Rather, a metric such as near_limit has a high cardinality because it has a descriptor key as a label for prometheus.

And there is currently no implementation such as allow-list, but I think it is a point that can be improved.

I'll be contributing code soon and listening to feedback on slack, github, and more.

[birdayz]

We could either make each metric configurable (on/off). In addition, anyone can use prometheus labeldrop to reduce cardinality. Do we need anything beyond that?