fluent
in_opentelemetry: add tag_from_uri support to http2 codepath
For some reason the new HTTP/2 server codepaths in the OpenTelemetry input plugin don't have any support for the tag_from_uri plugin option.
This PR adds the same v1_metrics / v1_logs / v1_traces tags that the tag_from_uri behaviour in the previous flb_downstream-based non-HTTP/2 codepath.
Strictly speaking, this is arguably not fully correct, as the "URI"s in the gRPC cases are more complex:
opentelemetry.proto.collector.metrics.v1.MetricsService/Exportopentelemetry.proto.collector.traces.v1.TracesService/Exportopentelemetry.proto.collector.logs.v1.LogsService/Export
However it's probably a good enough first step given that anyone trying to filter by telemetry signal type would already be using these v1_metrics / v1_logs / v1_traces tags, and it seems overly complex to have totally different longer tags just for gRPC input.
As I said in an earlier comment on #8734, the "proper" fix for this IMO is to switch to a tag_from_signaltype option or something like that. But this is more work and might count as a breaking config change. This PR at least brings the same behaviour as the non-HTTP/2 config case to the HTTP/2 config case.
"Next-step" fix for #8734 (previous PR #8881 only addressed cases where http2 plugin option was set to false).
Enter [N/A] in the box, if an item is not applicable to your change.
Testing Before we can approve your change; please submit the following in a comment:
- [x] Example configuration file for the change
service:
log_level: debug
pipeline:
inputs:
- name: opentelemetry
tag_from_uri: true
http2: true
outputs:
- name: 'stdout'
match: 'v1_logs'
cURL command for testing:
curl --header "Content-Type: application/json" --request POST --data '{"resourceLogs":[{"resource":{},"scopeLogs":[{"scope":{},"logRecords":[{"timeUnixNano":"1660296023390371588","body":{"stringValue":"{\"message\":\"dummy\"}"},"traceId":"","spanId":""}]}]}]}' http://localhost:4318/v1/logs
- [x] Debug log output from testing the change
[2024/07/01 10:20:54] [ info] [fluent bit] version=3.1.0, commit=5aab3eb02c, pid=16393
[2024/07/01 10:20:54] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2024/07/01 10:20:54] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/07/01 10:20:54] [ info] [cmetrics] version=0.9.1
[2024/07/01 10:20:54] [ info] [ctraces ] version=0.5.1
[2024/07/01 10:20:54] [ info] [input:opentelemetry:opentelemetry.0] initializing
[2024/07/01 10:20:54] [ info] [input:opentelemetry:opentelemetry.0] storage_strategy='memory' (memory only)
[2024/07/01 10:20:54] [debug] [opentelemetry:opentelemetry.0] created event channels: read=21 write=22
[2024/07/01 10:20:54] [debug] [downstream] listening on 0.0.0.0:4318
[2024/07/01 10:20:54] [ info] [input:opentelemetry:opentelemetry.0] listening on 0.0.0.0:4318
[2024/07/01 10:20:54] [debug] [stdout:stdout.0] created event channels: read=24 write=25
[2024/07/01 10:20:54] [ info] [sp] stream processor started
[2024/07/01 10:20:54] [ info] [output:stdout:stdout.0] worker #0 started
[2024/07/01 10:21:05] [debug] [input:opentelemetry:opentelemetry.0] attributes missing
[2024/07/01 10:21:05] [debug] [task] created task=0x7dec24023470 id=0 OK
[2024/07/01 10:21:05] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] v1_logs: [[1660296023.1698112429, {}], {"log"=>"{"message":"dummy"}"}]
[2024/07/01 10:21:05] [debug] [out flush] cb_destroy coro_id=0
[2024/07/01 10:21:05] [debug] [task] destroy task=0x7dec24023470 (task_id=0)
- [ ] Attached Valgrind output that shows no leaks or memory corruption was found
If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
- [ ] Run local packaging test showing all targets (including any new ones) build.
- [ ] Set
ok-package-testlabel to test for all targets (requires maintainer to do).
Documentation
- [ ] Documentation required for this feature
Backporting
- [ ] Backport to latest stable release.
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.
@edsiper @cosmo0920 I've fixed up the formatting of the commit messages / PR titles on this one. It should properly fix the issue in #8734
