HTTP 400 when sending log to ElasticSearch 8.0.0

[Belorus]

trafficstars

Versions: .NET Core 3.1 Serilog 2.9.0 Serilog.Sinks.ElasticSearch 8.1.0

Change in ElasticSearch 8.0 https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html

Code that needs change: If I set TypeName to null (so that it doesn't get into resulting JSON) - everything works like a charm on both 7.x and 8.x

https://github.com/serilog/serilog-sinks-elasticsearch/blob/f8bfaf86bc28a27b8259459baa9a75db7729e342/src/Serilog.Sinks.Elasticsearch/Sinks/ElasticSearch/ElasticsearchSinkState.cs#L75

Error screenshot:

[mivano]

Thanks for reporting. I have not used this version yet. Do you have a proposal on how to tackle this?

[orjan]

One way could possible be to specify the Elastic version in the configuration. That is easy to understand but make it mandatory would cause a breaking change.

Making it optional and detect the version if not specified by calling GET / could also be an option to set the parameter if not specified?

Another option would be to have a default version and if possible configure the version in runtime? https://github.com/uken/fluent-plugin-elasticsearch#verify_es_version_at_startup

[mivano]

We do have an detection of the version, so we can use that?

[orjan]

@Belorus would you mind try to set TypeName = null in the latest version 8.4.0

https://github.com/serilog/serilog-sinks-elasticsearch/blob/dev/CHANGES.md

[hastarin]

Silly me just updated our dev ELK stack to 8.0.0 without checking things would work.

I see this is also tracked via #375

At the moment it looks like updating all our apps configuration via code will be needed: TypeName = null, BatchAction = ElasticOpType.Create

[digger73]

I also ran into this problem and I'm reliant on the config files. However, it isn't possible to set TypeName to null. Is it possible to combine the use of configuraiton files and code config ?

[orjan]

@digger73 I'm not running Elastic any longer but if I'm not wrong, it must be set through code since it's not possible to deliberately set it to null trough configuration?

[digger73]

@orjan I came to the same conclusion, that was looking for the solution of using both the app.config and configuration of the sink in the code, however I haven't gotten this to work. When using app.config the elasticsearch sink is registered as a restricted sink and the options set through code is registered as an elasticsearch sink.