fluentd icon indicating copy to clipboard operation
fluentd copied to clipboard

Allow serving compresses responses via Prometheus plugin

Open Lusitaniae opened this issue 1 month ago • 0 comments

Is your feature request related to a problem? Please describe.

I'm using Prometheus plugin to export metrics based on logs ingested by fluentd.

In some servers these metrics can amount to millions of lines, which reach payload sizes of 50/60Mb and either reach the scraping limit (set by vmagent), or they failed to be transitted within the timeout period (10s)

Compressing data would be a clear winner to solve this problem

Describe the solution you'd like

Allow prometheus plugin to send compressed responses

Describe alternatives you've considered

Bumping payload limits / timeouts

Additional context

$ curl -s http://node1:9106/metrics | wc -c
37420840
$ curl -s http://node1:9106/metrics | gzip -c | wc -c
1877477

Cross posted from https://github.com/fluent/fluent-plugin-prometheus/issues/219 in case this is the right place to implement such feature

Lusitaniae avatar May 22 '24 18:05 Lusitaniae