fluent-bit-docs icon indicating copy to clipboard operation
fluent-bit-docs copied to clipboard

FluentBit Nats Output Plugin JSON format problems

Open benmutlu opened this issue 1 year ago • 0 comments

Hello, I am trying to ship following json message to NATS server with using FluentBit NATS output plugin.

My Input and configuration { "proxy_cache_status": "HIT", "timestamp": "2024-02-07T14:26:44+03:00", "remote_addr": "1.1.1.1", "remote_port": "1111", "request_addr": "1.1.1.1", "server_name": "test.com", "server_addr": "2.2.2.2", "server_port": "443", "body_bytes_sent": "1107221", "bytes_sent": "1109155", "body_received": "133", "request_time": "0.000", "status": "200", "request_uri": "/test.html", "request_param": "", "request_method": "GET", "upstream_addr": "", "upstream_response_time": "", "upstream_connect_time": "", "upstream_header_time": "", "upstream_status": "", "scheme": "https", "http_referrer": "https://googe.com", "tcpinfo_rtt": 22102, "tcpinfo_rttvar": 9038, "ssl_protocol": "TLSv1.3", "ssl_cipher": "TLS_AES_256_GCM_SHA384", "request_id": "12312313213asdasd", "role": "Edge", "http_range": "", "server_ip": "1.1.1.1", "account_type": "Test", "server_role": "Node", "http_x_forwarded_for": "5.31.6.128", "http_protocol": "HTTP/2.0", "http_user_agent": "Mozilla/5.0"}

FluentBit configuration [SERVICE] Flush 10 Log_Level debug Daemon off Parsers_File parsers.conf

[INPUT] Name tail Path /var/log/test.log Parser json Tag test.access.log Docker_Mode On Read_from_Head true

[OUTPUT] Name nats Match test.access.log Host 192.168.1.1 Port 4222 Retry_Limit 10

The problem is I am getting this message from NATS https://docs.fluentbit.io/manual/pipeline/outputs/nats this document verify that output should be similar like below, but I want to process messages with JSON format, so I need to write messages same like source, how can I do that ?

[1707293335.080561,{ "proxy_cache_status": "HIT", "timestamp": "2024-02-07T14:26:44+03:00", "remote_addr": "1.1.1.1", "remote_port": "1111", "request_addr": "1.1.1.1", "server_name": "test.com", "server_addr": "2.2.2.2", "server_port": "443", "body_bytes_sent": "1107221", "bytes_sent": "1109155", "body_received": "133", "request_time": "0.000", "status": "200", "request_uri": "/test.html", "request_param": "", "request_method": "GET", "upstream_addr": "", "upstream_response_time": "", "upstream_connect_time": "", "upstream_header_time": "", "upstream_status": "", "scheme": "https", "http_referrer": "https://googe.com", "tcpinfo_rtt": 22102, "tcpinfo_rttvar": 9038, "ssl_protocol": "TLSv1.3", "ssl_cipher": "TLS_AES_256_GCM_SHA384", "request_id": "12312313213asdasd", "role": "Edge", "http_range": "", "server_ip": "1.1.1.1", "account_type": "Test", "server_role": "Node", "http_x_forwarded_for": "5.31.6.128", "http_protocol": "HTTP/2.0", "http_user_agent": "Mozilla/5.0"}}]]

benmutlu avatar Feb 07 '24 12:02 benmutlu