Send data to another system and load to elastic

[adm9000-3]

I am trying to establish a chain: rsyslog -> syslog-ng (another server) -> elasticsearch. elasticsearch working on localhost of another server (with syslog-ng). Thats why i cant to use rsyslog elastic module which works fine (in documentation).

With configuration below i am making an index and data, but unable to read the index. I guess there is a misconfiguration.

I am read documentation of Tlog, but there is no info about situations when i have to send data to another system. Any ideas? Thanks.

---

rsyslog.conf: $MaxMessageSize 3k

template(name="tlog" type="list") { constant(value="{") property(name="timegenerated" outname="timestamp" format="jsonf" dateFormat="rfc3339") constant(value=",") property(name="msg" regex.expression="{\(.*\)" regex.submatch="1") constant(value="\n") }

if $!_UID == "998" then {

action(type="omfwd" Target="10.0.3.14" Port="25007" Protocol="udp" Device="ens192" template="tlog") ~ }

syslog-ng.conf:

source s_fex_tlog { udp(port(25007) log-iw-size(1000000) log-fetch-limit(100000)); };

destination d_fex_tlog { http(url("http://127.0.0.1:9200/_bulk") method("POST") batch-lines(10000) batch-timeout(60000) batch-bytes(100Mb) headers("Content-Type: application/x-ndjson") body-suffix("\n") body("{"index":{"_index":"tlog-${YEAR}.${MONTH}.${DAY}","_type":"_doc"}}\n$(format-json --scope nv-pairs --exclude DATE --key ISODATE)\n") persist-name(d_fe1_tlog) ); };

log { source (s_fex_tlog); destination (d_fex_tlog); };

[justin-stephenson]

HI @adm9000-3 sorry I cannot really help with the syslog-ng configuration, but maybe other tlog users will be able to provide input here if they have a working configuration.

If the message is getting stored on both non-working and working servers, can you compare the fields in elasticsearch to see if anything is different?