fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

ability to ingest Prometheus metrics from a file

Open peterhack opened this issue 6 months ago • 2 comments

Is your feature request related to a problem? Please describe. Cannot ingest prometheus metrics from a file/files in any format

Describe the solution you'd like Looking for a way to ingest prometheus metrics from a file / files, in any format - we'll control the thing that generates the files, we'd just rather run it as a cron or similar instead of an always-on listener that fluentbit would scrape.

Describe alternatives you've considered We didn't find any existing "get metrics from files" plugin, so I thought we could explore using logs_to_metrics for this. However, https://docs.fluentbit.io/manual/pipeline/filters/log_to_metrics says: This filter is an experimental feature and is not recommended for production use. Configuration parameters and other capabilities are subject to change without notice.

Does this mean I should avoid logs_to_metrics for now, or is it pretty stable and likely to continue to work? Is there another option? Maybe a better option?

peterhack avatar Jun 17 '25 15:06 peterhack

How about tail then logs-to-metrics as you say? I think the docs are just cautioning you and probably need updating now the feature has been in for a while.

patrick-stephens avatar Jun 18 '25 14:06 patrick-stephens

This would be a useful feature in its own right.

However, if you think logs_to_metrics is a stable feature, then yes please get the documentation updated.

g-cos avatar Jun 18 '25 15:06 g-cos

Feel free to send a docs PR, it's all OSS :)

patrick-stephens avatar Jun 20 '25 11:06 patrick-stephens

That doesn't help - I'm not the one who knows what it should say, I'm asking about what it currently says. I should not be the one to send a PR, someone who knows that the current doc is misleading and who knows what it really should say needs to do that.

g-cos avatar Jun 20 '25 13:06 g-cos

@peterhack here you go: https://github.com/fluent/fluent-bit/pull/10641

edsiper avatar Jul 23 '25 23:07 edsiper

https://github.com/fluent/fluent-bit/pull/10641 has been merged, it will be part of v4.0.6

edsiper avatar Jul 24 '25 17:07 edsiper

Thank you, this is great, we'll try it soon!

Is there a corresponding PR in https://github.com/fluent/fluent-bit-docs ?

g-cos avatar Jul 24 '25 19:07 g-cos