fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

Metrics input processor documenation

Open jenschude opened this issue 1 year ago • 1 comments

Is your feature request related to a problem? Please describe.

When scraping metrics I would like to group/tag specific metrics when they have a specific label e.g. tenant. The filter options will not work as they only apply to logs. Atm the processor pipeline seems to be the way to go but going through documentation and sources I don't see how to write a processor able for usage with metrics. I can see that metrics as well as trace processors are supported but there is no documentation and/or tests about it.

Describe the solution you'd like

  • add documentation for a Lua script to process metrics for input as well as output.

  • document the C, Rust, Go interface for metrics and trace processors.

Describe alternatives you've considered

OpenTelemetry collector comes with an groupsByAttrs processor. Something similar could be added to fluentbit directly.

https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/groupbyattrsprocessor

Additional context

jenschude avatar Mar 05 '24 09:03 jenschude

I think https://github.com/fluent/fluent-bit/pull/8526 was in a similar vein so @cosmo0920 may have some pointers. A docs PR would be appreciated as well @jenschude.

patrick-stephens avatar Mar 06 '24 15:03 patrick-stephens

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

github-actions[bot] avatar Aug 22 '24 01:08 github-actions[bot]

Processor_metrics_selector could work for filtering metrics type of events and its documentation is already up: https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/processors/metrics-selector.md

cosmo0920 avatar Aug 22 '24 07:08 cosmo0920