OpenTelemetry Plugins should have a way to modify Resource and Scope for metrics and traces

[nortenorte]

Is your feature request related to a problem? Please describe. Originally this was submitted in #8206 and was implemented for logs only in #8989. We would like to be able to use the same for traces and metrics as well. #8206 was closed even it was only partially implemented.

Describe the solution you'd like See #8206.

Describe alternatives you've considered N/A

Additional context We would like to be able to modify for the same reasons as for logs.

[ezalorsara]

Hi any news to this. Thank you

[thediveo]

that's a roadblock for proposing fluentbit as an otel collector in our company.

[edsiper]

@nortenorte @ezalorsara @thediveo in the docs while not being very explicit, Trace is already supported, check the context where the processor can work:

https://docs.fluentbit.io/manual/pipeline/processors/content-modifier#contexts

we will keep this issue open until we improve the docs (@eschabell interested ? :) ) and finalize the support for Metrics contexts

[ardichoke]

I would note that the existing implementation is a bit incomplete. Unless I'm misreading the documentation, it seems that it can only apply these modifiers to an input. It would be extremely useful to be able to add Resource and Scope information on a per-output basis, or using a filter.

The specific use case I have is in k8s. I am working with an existing fluent-bit deployment which is collecting log data from all running pods, then uses the kubernetes filter to enrich the data.

What I would like to be able to do is take the data from the kubernetes filter and put it into the resource portion of the envelope, but I cannot because that can only be added through a processor on an input.

I eventually worked around this for the time being using the transform processor in OTel to manually move the metadata from log.attributes into resource.attributes, and enabled the flatten_data feature. This is less than ideal though as flatten_data incurs a non-negligible performance cost. It would be much more efficient to be able to put the data in the right place on the fluent-bit side of the pipeline.

[thediveo]

We've switched to the OTel collector and not looked back since then, very satisfied with the OTel collector.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.