serilog-sinks-elasticsearch icon indicating copy to clipboard operation
serilog-sinks-elasticsearch copied to clipboard

New version 9.0.3 doesn't work with Logstash

Open andrei-gutierrez opened this issue 1 year ago • 1 comments

Does this issue relate to a new feature or an existing bug?

  • [x] Bug
  • [ ] New Feature

What is the target framework and operating system? See target frameworks & net standard matrix.

  • [x] netCore 2.0
  • [ ] netCore 1.0
  • [ ] 4.7
  • [ ] 4.6.x
  • [ ] 4.5.x

New version of the sink doesn't work with logstash, something changed between 9.0.0 and the actual that is unable to read from logstash. Could it be the formatting? This is my pipeline I used to read the sink:

input{
  http{
    port => "5045"
    additional_codecs => { "application/json" => "es_bulk" }
}

Version 9.0.3

Please describe the current behavior? Shows nothing in logstash, no errors, no warning, looks like it is being dropped all the package

Please describe the expected behavior? Receive the logs as it was in 8.4.1

andrei-gutierrez avatar Jun 19 '23 07:06 andrei-gutierrez

This is also not working using net cat:

  1. Setup your netcat:
$ nc -lp 1234
  1. Configure the sink:
{"args": {
   "indexFormat": "mylogs"
   "nodeUris": "http://localhost:1234"
 }
}
  1. Test on a working version I get this:
$ nc -lp 1234
POST /_bulk HTTP/1.1
Accept: application/json
Content-Type: application/json
User-Agent: elasticsearch-net/7.8.1+aed95253ed9d86b6e22107b3b4a6e2496c206cd4 (Microsoft Windows 10.0.19044; .NET Framework 4.8.4614.0; Elasticsearch.Net)
Host: localhost:1234
Content-Length: 2395
Connection: Keep-Alive

{"index":{"_type":"logevent","_index":"mylogs"}}
{"@timestamp":"2023-06-19T11:21:47.5933802+02:00","level":"Information","messageTemplate":"    C:\\Users\\OneDrive\\Documents\\Application\\Plugins","message":"    C:\\Users\\OneDrive\\Documents\\Application\\Plugins","fields":{"ActivityId":"00000000-0000-0000-0000-000000000000","TraceSource":"Application.exe","TraceEventType":"Information","TraceEventId":0,"SourceContext":"SerilogTraceListener.SerilogTraceListener","Context":"
[...]
  1. Test on a non working version I get this:
$ nc -lp 1234
GET / HTTP/1.1
Accept: application/json
Content-Type: application/json
User-Agent: elasticsearch-net/7.17.5+34f1b0b42ae4e0227fb1ad9ebeb7ae3406a998ec (Microsoft Windows 10.0.19044; .NET Framework 4.8.4614.0; Elasticsearch.Net)
x-elastic-client-meta: es=7.17.5,net=4.8.0,t=7.17.5,a=1,wr=4.8.0
Host: localhost:1234
Connection: Keep-Alive

$

Notice it is empty on step 4 and bring back to the console.

andrei-gutierrez avatar Jun 19 '23 09:06 andrei-gutierrez