beats
beats copied to clipboard
7.17 filebeat `elasticsearch.slowlog` ingested log entry has full json log as message
When using filebeat to ingest ES slowlogs, the resulting document does not have the correct message. Instead the message contains the full JSON log entry:
Reproduction:
- Version
7.17.21
for Elasticsearch and Filebeat - Setup filebeat to ingest slowlogs using the
elasticsearch.slowlog
module (For example by using the log+metrics feature in elastic cloud)
Here an example of a log entry as it appears in the ES log file:
{"type": "index_search_slowlog", "timestamp": "2024-06-13T11:54:45,125Z", "level": "WARN", "component": "i.s.s.query", "cluster.name": "13fd6fa94ab840c088e86a8cd8faa3b8", "node.name": "instance-0000000000", "message": "[kibana_sample_data_ecommerce][0]", "took": "5.5ms", "took_millis": "5", "total_hits": "0+ hits", "types": "[]", "stats": "[]", "search_type": "QUERY_THEN_FETCH", "total_shards": "1", "source": "{\"size\":500,\"query\":{\"bool\":{\"filter\":[{\"range\":{\"order_date\":{\"from\":\"2024-06-13T11:39:45.023Z\",\"to\":\"2024-06-13T11:54:45.023Z\",\"include_lower\":true,\"include_upper\":true,\"format\":\"strict_date_optional_time\",\"boost\":1.0}}}],\"adjust_pure_negative\":true,\"boost\":1.0}},\"version\":true,\"_source\":false,\"stored_fields\":\"*\",\"fields\":[{\"field\":\"*\",\"include_unmapped\":true},{\"field\":\"customer_birth_date\",\"format\":\"strict_date_optional_time\"},{\"field\":\"order_date\",\"format\":\"strict_date_optional_time\"},{\"field\":\"products.created_on\",\"format\":\"strict_date_optional_time\"}],\"script_fields\":{},\"sort\":[{\"order_date\":{\"order\":\"desc\",\"unmapped_type\":\"boolean\"}}],\"track_total_hits\":-1,\"highlight\":{\"pre_tags\":[\"@kibana-highlighted-field@\"],\"post_tags\":[\"@/kibana-highlighted-field@\"],\"fragment_size\":2147483647,\"fields\":{\"*\":{}}}}", "id": "b060ba95-ddea-4aa2-8a36-82a7a36b1564", "cluster.uuid": "UgI0rr5lSqeQtW98mENVxQ", "node.id": "BbJ5ZoU7QE2Q-U6fMAXgng" , "trace.id": "e738203f84d7190e649598ee3a54152d" }