Lua Filter record can't see __internal__.group_attributes.resource.attributes coming via Opentelemetry input plugin

[KoushikBSK]

Lua Filter record can't see internal.group_attributes.resource.attributes coming via Opentelemetry input plugin.

I can see internal.group_attributes.resource.attributes via stdout out plugin , before lua filter.

When I query record inside lua filter it only has "message" key. nothing else.

output plugin after lua scripts shows log attributes & "message" etc, but no resource attributes.

[ryn9]

From what I gather, there currently is not yet a way to interface with the metadata/internal-data with lua yet.

The _internal_ stuff you are seeing appears to be just a display/output function: https://github.com/fluent/fluent-bit/pull/10282

Regarding the state of metadata access in lua, I would have a look here: https://github.com/fluent/fluent-bit/pull/9702

[ryn9]

This one I just reported may be an issue as well: https://github.com/fluent/fluent-bit/issues/10442

[KoushikBSK]

Thanks a lot for looking at this , yes I went through those. I got confused by reading this and thought new lua processor is already created.

=========================

update: Jan 09, 2024

We will add this functionality as opt-in for v4, in the meanwhile a new processor is created as described in the updated info of Dec 24, 2024.

[KoushikBSK]

any other processor or filter can use those " internal" :"group_attributes" ?

[edsiper]

WIP: Lua filter enhancements: https://github.com/fluent/fluent-bit/pull/10457

[edsiper]

fixed in https://github.com/fluent/fluent-bit/pull/10457

[KoushikBSK]

fixed in #10457

Thanks a lot Eduardo, I can confirm it's working, I can now access Opentelemetry Group/metadata. Just to double confirm the record is modifiable right , not read-only ?