Default LogsDB value for `ignore_dynamic_beyond_limit`

[salvatore-campagna]

When ingesting logs, it's important to ensure that documents are not dropped due to mapping issues, also when dealing with dynamically mapped fields. Elasticsearch provides two key settings that help manage the total number of field mappings and handle situations where this limit might be exceeded:

1. index.mapping.total_fields.limit: This setting defines the maximum number of fields allowed in an index. If this limit is reached, any further mapped fields would cause indexing to fail.

2. index.mapping.total_fields.ignore_dynamic_beyond_limit: This setting determines whether Elasticsearch should ignore any dynamically mapped fields that exceed the limit defined by index.mapping.total_fields.limit. If set to false, indexing will fail once the limit is surpassed. However, if set to true, Elasticsearch will continue indexing the document but will silently ignore any additional dynamically mapped fields beyond the limit.

To prevent indexing failures due to dynamic mapping issues, especially in logs where the schema might change frequently, we change the default value of index.mapping.total_fields.ignore_dynamic_beyond_limit from false to true in LogsDB. This change ensures that even when the number of dynamically mapped fields exceeds the set limit, documents will still be indexed, and additional fields will simply be ignored rather than causing an indexing failure.

This adjustment is important for LogsDB, where dynamically mapped fields may be common, and we want to make sure to avoid documents from being dropped.