ecs-dotnet
ecs-dotnet copied to clipboard
elastic
[BUG] Synthetic Source Compatibility Error in Elastic.Serilog.Sinks 9.x
ECS integration/library project(s) (e.g. Elastic.CommonSchema.Serilog): Elastic.Serilog.Sinks (Elastic.CommonSchema.Elasticsearch)
ECS schema version: 9.0.0
ECS .NET assembly version: 9.0.0
Elasticsearch version: 8.13.2
.NET framework / OS: .NET Core 9 / Windows Server 2020
Description of the problem, including expected versus actual behavior:
After upgrading to Elastic.Serilog.Sinks 9.x, the library attempts to create component templates with the synthetic_source_keep parameter, which is not supported in some Elasticsearch 8.13.2 configurations, despite the version number suggesting compatibility.
Error Message:
Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.Exception: Failure to create component template `ecs_9.0.0_base` for logs-srv-*: Invalid Elasticsearch response built from a unsuccessful (400) low level call on PUT: /_component_template/ecs_9.0.0_base
Exception: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""
# Audit trail of this API call:
- [1] BadResponse: Node: http://localhost:9200/ Took: 00:00:00.0190869
# OriginalException: Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""
# Request:
{
"_meta": {
"documentation": "https://www.elastic.co/guide/en/ecs/current/ecs-base.html",
"ecs_version": "9.0.0"
},
"template": {
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"labels": {
"type": "object"
},
"message": {
"type": "match_only_text"
},
"tags": {
"ignore_above": 1024,
"synthetic_source_keep": "none",
"type": "keyword"
}
}
}
}
}
# Response:
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]","caused_by":{"type":"mapper_parsing_exception","reason":"unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]"}},"status":400}
---> Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""
--- End of inner exception stack trace ---
at Elastic.Ingest.Elasticsearch.ElasticsearchChannelBase`2.PutComponentTemplate(BootstrapMethod bootstrapMethod, String name, String body)
at Elastic.Ingest.Elasticsearch.CommonSchema.EcsDataStreamChannel`1.BootstrapElasticsearch(BootstrapMethod bootstrapMethod, String ilmPolicy)
at Elastic.Serilog.Sinks.ElasticsearchSink`1..ctor(ElasticsearchSinkOptions`1 options)
at Elastic.Serilog.Sinks.ElasticsearchSink..ctor(ElasticsearchSinkOptions options)
at Elastic.Serilog.Sinks.ConfigSinkExtensions.Elasticsearch(LoggerSinkConfiguration loggerConfiguration, BootstrapMethod bootstrapMethod, ICollection`1 nodes, Boolean useSniffing, String dataStream, String ilmPolicy, String apiKey, String username, String password, Nullable`1 includeHost, Nullable`1 includeActivity, Nullable`1 includeProcess, Nullable`1 includeUser, ICollection`1 filterProperties, Nullable`1 maxRetries, Nullable`1 maxConcurrency, Nullable`1 maxInflight, Nullable`1 maxExportSize, Nullable`1 maxLifeTime, Nullable`1 fullMode, Uri proxy, String proxyUsername, String proxyPassword, String fingerprint, Boolean debugMode, LoggingLevelSwitch levelSwitch, LogEventLevel restrictedToMinimumLevel)
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
--- End of inner exception stack trace ---
at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at Serilog.Settings.Configuration.ConfigurationReader.CallConfigurationMethods(ILookup`2 methods, IReadOnlyCollection`1 configurationMethods, Object receiver)
at Serilog.Settings.Configuration.ConfigurationReader.ApplySinks(LoggerConfiguration loggerConfiguration)
at Serilog.Settings.Configuration.ConfigurationReader.Configure(LoggerConfiguration loggerConfiguration)
at Serilog.Configuration.LoggerSettingsConfiguration.Settings(ILoggerSettings settings)
at Serilog.ConfigurationLoggerConfigurationExtensions.Configuration(LoggerSettingsConfiguration settingConfiguration, IConfiguration configuration, ConfigurationReaderOptions readerOptions)
at Program.<Main>$(String[] args)
at Program.<Main>(String[] args)
Steps to reproduce:
- Have an Elasticsearch 8.13.2 cluster (free version)
- Install Elastic.Serilog.Sinks 9.0.0
