Felix Barnsteiner

> using the presence of "ecs.version" (along with the other 3 required fields) to pre-check that a log line is an ECS log record Ah, right. That's not only useful...

> I would also assume that `event.dataset` and `data_stream.dataset` will coexist. That's how I specified it. Both `event.dataset` and `data_stream.dataset` will be set by the loggers. But starting with version...

I really think it's the responsibility of the "other tool" to add the defaults. It already needs to add metadata about the host, pod, container, etc anyway.

@ruflin is adding `event.dataset` as an alias for `data_stream.dataset` something you're considering in the index templates created by Elastic Agent? Should ecs loggers add both `event.dataset` and `data_stream.dataset fields? The...

> Elastic Agent does not have any templates, I assume you are referring to the templates shipped by Elasticsearch? I was referring to the templates shipped with Filebeat that are...

Let's try to get this one over the finishing line. > Lets try to discuss first the end goal and then work backwards from there what we can do to...

I hit an issue when testing this end-to-end with https://github.com/elastic/ecs-logging-java/pull/124 and Elastic Agent 7.11.2: The `data_stream.dataset` field in the log files were overridden by the `Dataset name` configured in the...

I've created an issue: https://github.com/elastic/beats/issues/24683 I'll mark this PR as blocked, at least until we have a consensus that this should be implemented in the near to mid-term.

It definitely does. Reading this thread is almost nostalgic 😄. This thread was the reason I got pulled in more and more into routing and it seems https://github.com/elastic/elasticsearch/pull/76511 is finally...

Functionally, the PR looks good to me. The only missing piece is the data stream routing. Maybe a test case with complex (map, list) attributes could be useful. I'll defer...