set_fields() parameter to adjust for special characters in field name

[tmacgbay]

Issue to be solved

Data processed by pipeline rule function key_value() might contain spaces in the key portion. This makes the key data fail for set_fields() - example:

vpn type=device level vpn

It is possible to use regex_replace() to replace the spaces but it affects the key and the value (unless a regex guru shows me otherwise!)

Feature to resolve issue

Add an optional parameter to set_fields() to convert special characters in the key portion.

Why?

Setting this, perhaps with a default of "_" at set_fields() will reduce bad data coming from key_value() and grok() when set_fields() is used. Currently the failure happens silently (nothing in graylog logs).

Environment

• Graylog Version: 3.1.3
• Elasticsearch Version: 6.8.5
• MongoDB Version: 4.0.13
• Operating System: Ubuntu 19.10 eoan
• Browser version: chrome 78.0.3904.108