fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

How to extract otlp.attributes?

Open psyhomb opened this issue 7 months ago • 1 comments

I'm sending OpenTelemetry events using otelgen tool, and I'd like to filter them based on namespace. Does anyone know how can I extract k8s.namespace.name and rewrite it as a tag?

[2] otlp: [[1748383193.18446744073409639198, {"otlp"=>{"observed_timestamp"=>1748383193676245469, "timestamp"=>1748383193676245239, "severity_number"=>1, "severity_text"=>"Trace", "attributes"=>{"wor
ker_id"=>"0", "service.name"=>"otelgen", "trace_id"=>"bda0306f05bde838775df7254e3b6260", "span_id"=>"d65da4f7a558d1d1", "trace_flags"=>"01", "phase"=>"processing", "http.method"=>"PUT", "http.status_
code"=>400, "http.target"=>"/api/v1/resource/0", "k8s.pod.name"=>"otelgen-pod-9d73fcd0", "k8s.namespace.name"=>"default", "k8s.container.name"=>"otelgen"}, "trace_flags"=>0}}], {"message"=>"Log 0: Tr
ace phase: processing"}]

I tried something like this but it doesn't work:

[FILTER]
    Name     rewrite_tag
    Match    otlp
    Rule     $otlp['attributes']['k8s.namespace.name']  ^(.*)$  ns.$1  false

I also tried using modify filter and copy the key first but that doesn't work either:

[INPUT]
    Name opentelemetry
    Listen 127.0.0.1
    Port 4318
    tag_from_uri false
    tag otlp

[FILTER]
    Name     modify
    Match    otlp
    Copy     $otlp['attributes']['k8s.namespace.name'] k8s_namespace_name

[FILTER]
    Name     rewrite_tag
    Match    otlp
    Rule     k8s_namespace_name  ^(.*)$  ns.$1  false

[OUTPUT]
    Name     stdout
    Match    ns.default

Log output:

[2025/05/27 23:54:53] [debug] [filter:modify:modify.0] Rule COPY $otlp['attributes']['k8s.namespace.name'] TO k8s_namespace_name : No keys matching $otlp['attributes']['k8s.namespace.name'] found, no
t applying rule

OpenTelemetry input seems to be dropping the otlp (metadata) object.

psyhomb avatar May 27 '25 22:05 psyhomb

WIP: Lua filter enhancements: https://github.com/fluent/fluent-bit/pull/10457

edsiper avatar Jun 09 '25 23:06 edsiper

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

github-actions[bot] avatar Sep 08 '25 02:09 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Oct 25 '25 02:10 github-actions[bot]