Lua filter before opentelemetry_envelope does not work
Bug Report
Describe the bug I'm collecting logs in syslog format, then adding custom fields to opentelemetry attributes before sending them to opentelemetry. I need to extract the host field from the syslog message and map it to the opentelemetry attribute host.name. To achieve this, I'm using a Lua script to enrich messages for OpenTelemetry, (otel_resource_attributes only supports text values, as I understand it)
To Reproduce
- place Lua filter before 'opentelemetry_envelope'
[0] bmclogs: [1748878869.000000000, {}, {"pri"=>"174", "time"=>"Jun 2 15:41:09", "host"=>"myhost", "ident"=>"maint_debug_mod", "message"=>"2025-06-02 15:41:09 Redfish,user@myhost,redfish,User(user) authentication failed"}]
[2025/06/02 15:41:15] [debug] [processor] no condition set for processor unit (pu=opentelemetry_envelope)
[0] bmclogs: [1748878869.000000000, {}, {"pri"=>"174", "attributes"=>{"resource.id"=>"myresourceid", "service.name"=>"mycollector", "product.code"=>"myproduct", "host.name"=>"myhost"}, "time"=>"Jun 2 15:41:09", "ident"=>"maint_debug_mod", "host"=>"myhost", "message"=>"2025-06-02 15:41:09 Redfish,user@myhost,redfish,User(user) authentication failed"}]
Expected behavior The Lua script should add the required fields, allowing me to reference them via logs_attributes_metadata_key.
Your Environment
- Version used: Fluent Bit v4.0.1
- Operating System and version: Ubuntu 22.04.5 LTS
- Configuration:
pipeline:
inputs:
- name: syslog
tag: bmclogs
listen: ::0
parser: syslog-rfc3164
mode: tcp
port: 5140
processors:
logs:
- name: stdout
- name: lua
code: |
function cb_modify(tag, timestamp, record)
local service_name = "mycollector"
local product_code = "myproduct"
local host_name = record["host"] or "unknown"
local resource_id = "myresourceid"
local new_record = record
new_record["attributes"] = {
["resource.id"] = resource_id,
["service.name"] = service_name,
["product.code"] = product_code,
["host.name"] = host_name
}
return 1, timestamp, new_record
end
call: cb_modify
- name: opentelemetry_envelope
outputs:
- name: opentelemetry
match: bmclogs
host: collector.local
port: 4318
logs_uri: /v1/logs
tls: true
tls.verify: true
log_response_payload: true
logs_attributes_metadata_key: $attributes
WIP: Lua filter enhancements: https://github.com/fluent/fluent-bit/pull/10457
fixed in https://github.com/fluent/fluent-bit/pull/10444