fluentd
fluentd copied to clipboard
Allow serving compresses responses via Prometheus plugin
Is your feature request related to a problem? Please describe.
I'm using Prometheus plugin to export metrics based on logs ingested by fluentd.
In some servers these metrics can amount to millions of lines, which reach payload sizes of 50/60Mb and either reach the scraping limit (set by vmagent), or they failed to be transitted within the timeout period (10s)
Compressing data would be a clear winner to solve this problem
Describe the solution you'd like
Allow prometheus plugin to send compressed responses
Describe alternatives you've considered
Bumping payload limits / timeouts
Additional context
$ curl -s http://node1:9106/metrics | wc -c
37420840
$ curl -s http://node1:9106/metrics | gzip -c | wc -c
1877477
Cross posted from https://github.com/fluent/fluent-plugin-prometheus/issues/219 in case this is the right place to implement such feature