es: Logstash_Prefix_Key with static prefix

[jplitza]

trafficstars

Bug Report

Describe the bug

Specifying the Logstash_Prefix_Key option with a static prefix (like containers-$kubernetes['namespace_name']) doesn't work, although they should be valid according to docs.

To Reproduce

• Rubular link if applicable:
• Example log message if applicable:

[2024/04/29 14:11:31] [error] [record accessor] syntax error, unexpected '$', expecting end of file at '$containers-$kubernetes['namespace_name']'
[2024/04/29 14:11:31] [error] [output:es:es.0] invalid logstash_prefix_key pattern '(null)'
[2024/04/29 14:11:31] [error] [output:es:es.0] cannot initialize plugin
[2024/04/29 14:11:31] [error] [output] failed to initialize 'es' plugin
[2024/04/29 14:11:31] [error] [engine] output initialization failed

• Steps to reproduce the problem: See in description above.

Expected behavior

The log is inserted into the index "containers-default" if the log came from a container in the namespace "default" and the kubernetes filter plugin added the kubernetes.namespace_name

Screenshots

Your Environment

• Version used:
• Configuration:
• Environment name and version (e.g. Kubernetes? What version?):
• Server type and version:
• Operating System and version:
• Filters and plugins:

Additional context

This is the relevant code line, which only checks if the first character of the Logstash_Prefix_Key value is $ and prepends $ otherwise, while (IMO) it should check if the value contains a $: https://github.com/fluent/fluent-bit/blob/9c7a4bf20a6949eaa2f60f1d6635707d0e2e6f13/plugins/out_es/es_conf.c#L329

[patrick-stephens]

A workaround is just to create another key with your prefix using another filter then use that.

[jplitza]

@patrick-stephens Could you maybe point me towards the right filter to achieve that? We are currently using Lua for that purpose, but that causes trouble with type conversions (#3343 to be precise, and the config key that was introduced there is no suitable workaround for us)

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[jplitza]

This issue is still relevant and even has a corresponding PR.