influxdata
Allow loki output to send raw messages
Use Case
I frequently add new apps that use JSON logging, and I have to pass the logs through a processor plugin in order to access the structured fields in Loki. It would be nice to just look at the message and have the metadata show when clicking on the message, instead of viewing the message and other metadata as unordered key value pairs.
Expected behavior
I would like there to be an option to set a message field in the loki output plugin then have all the option to either drop extra fields or have them sent to loki as labels.
Actual behavior
Option to disable logfmt is missing.
Additional info
No response
IIRC all Telegraf metric tags are sent as labels to Loki. So you could parse the JSON in Telegraf into a metric with the expected labels added as tags? I'm not sure what you are asking for?
I'm currently doing by either having the app log to local disk and having the tail plugin read it as json, or using the syslog input with a processor, but I would like to avoid having to do this for every app.
Not sure I get the target setting. Are you asking on passing through the content of a file / of each line in the file to Loki in a unmodified manner?
Could you make up an example of your input data and what you expect Telegraf to do with it, i.e. what is the expected Telegraf metric?
would it be possible to have the message field appear as the log line then have everything else as field
log line: "default message"
labels: source="ipahealthcheck.ipa.files" check="IPAFileCheck" result="SUCCESS" uuid="5660721c-a56f-4a22-ad77-278f434699c2" when="20241112100007Z" duration="0.519653" kw_key="_var_log_pki_pki-tomcat_ca_debug.2023-09-25.log_mode" kw_type="mode" kw_path="/var/log/pki/pki-tomcat/ca/debug.2023-09-25.log" order="2"namepass="<nil>"
instead of it being unordered key value pairs like this
source="ipahealthcheck.ipa.files" check="IPAFileCheck" result="SUCCESS" uuid="5660721c-a56f-4a22-ad77-278f434699c2" when="20241112100007Z" duration="0.519653" kw_key="_var_log_pki_pki-tomcat_ca_debug.2023-09-25.log_mode" kw_type="mode" kw_path="/var/log/pki/pki-tomcat/ca/debug.2023-09-25.log" order="2" message="default message" namepass="<nil>"