not abel to map nested field in logtrail.json file

[amiya-elear]

I am using filebeat to send data to ES. I am creating new raw field as prefix log like log.appname, log.messages etc but when i am try to map those field in logtrail.json its not giving any result logtrail version: logtrail-7.4.0-0.1.31.zip kibana version: kibana 7.4.0

logtrail.json { "version" : 2, "index_patterns" : [ { "es": { "default_index": "api-access-*" }, "tail_interval_in_seconds": 1, "nested_objects" : true, "es_index_time_offset_in_seconds": 0, "display_timezone": "local", "display_timestamp_format": "MMM DD HH:mm:ss", "max_buckets": 500, "default_time_range_in_days" : 0, "max_hosts": 100, "max_events_to_keep_in_viewer": 5000, "default_search": "", "fields" : { "mapping" : { "timestamp" : "log.timestamp", "hostname" : "host.name", "message": "log.messages" }, "message_format": " {{{log.messages}}}" }, "color_mapping" : { "field": "log.loglevel", "mapping": { "user.info": "#FF0000", "user.err": "#FFEF96", "user.warn": "#B5E7A0" } } } ] }

my kibana json file

"_index": "api-access-2019.10.18", "_type": "doc", "_id": "seSA3W0BrWb7gISVkILP", "_version": 1, "_score": null, "_source": { "@timestamp": "2019-10-18T06:13:43.591Z", "beat": { "hostname": "localhost", "version": "6.5.4", "name": "localhost" }, "host": { "name": "localhost" }, "log": { "gw_timestamp": "Oct 17 07:17:15", "gateway": "gateway", "loglevel": "user.warn", "appname": "xxxxx[1131]:", "messages": "xx():355: Deallocated pointer: 0x9cc2b0" }, "source": "/home/preetimuttagi/Desktop/messages", "offset": 296790233, "message": "Oct 17 07:17:15 gateway user.warn xxxx[1131]: xx():355: xx pointer: 0x9cc2b0", "input": { "type": "log" }, "prospector": { "type": "log" } }, "fields": { "@timestamp": [ "2019-10-18T06:13:43.591Z" ] }, "sort": [ 1571379223591 ] }

and color_mapping also not mapping log.loglevel field different log type not able to show only log.messages field in logtrail

kindly help thanks in advance

[sivasamyk]

@amiya-elear I assume you are using latest version of logtrail. If yes, you need to configure keyword_suffix parameter to raw as mentioned in README.

keyword_suffix - Specifies the keyword suffix to be appended for hostname & program fields. Set it to "" to not append any suffix. If not specified (undefined) logtrail will append keyword.

[amiya-elear]

Thanks for your replay . I am using logtrail couple of days i observe that if i do live log trail in kibana it's take huge amount of ram if my log size is more . Is there any way to control it .

[sivasamyk]

You mean the chrome instance running kibana or the ES instance? Can you share your logtrail.json. Also can you please let me know approximately how events every live tail request might fetch? In case of live tail kibana sends a request every 10s to fetch the events in last 10 seconds

[amiya-elear]

I install logtrail as plugin with kibana in cloud kibana using ES instance to fetch data . when i am doing log trailing in kibana its taking more ram.

sample event : timestamp:app_name log-level log_messages i kept "tail_interval_in_seconds: 4" because if i keep 10 second its taking time to fetch data from ES i need faster access of log.

{ "version" : 2, "index_patterns" : [ { "es": { "default_index": "gw-log-", "allow_url_parameter": true }, "tail_interval_in_seconds": 4, "nested_objects" : true, "es_index_time_offset_in_seconds": 0, "display_timezone": "local", "display_timestamp_format": "MMM DD HH:mm:ss", "max_buckets": 500, "default_time_range_in_days" : 0, "max_hosts": 100, "max_events_to_keep_in_viewer": 5000, "default_search": "", "fields" : { "mapping" : { "timestamp" : "@timestamp", "hostname" : "host.name", "message": "message" } }, "color_mapping" : { "field": "log.loglevel", "mapping": { "user.info": "#0000FF", "user.err": "#FF0000", "user.warn": "#FFFF00", "user.emerg": "#FFCCCB", "user.notice": "#00FFFF", "user.debug": "#008000", "user.crit": "#FF00FF" } } }, { "es": { "default_index": rbp-log-", "allow_url_parameter": true }, "tail_interval_in_seconds": 4, "nested_objects" : true, "es_index_time_offset_in_seconds": 0, "display_timezone": "local", "display_timestamp_format": "MMM DD HH:mm:ss", "max_buckets": 500, "default_time_range_in_days" : 0, "max_hosts": 100, "max_events_to_keep_in_viewer": 5000, "default_search": "", "fields" : { "mapping" : { "timestamp" : "@timestamp", "hostname" : "host.name", "message": "message" } }, "color_mapping" : { "field": "log.loglevel", "mapping": { "user.info": "#0000FF", "user.err": "#FF0000", "user.warn": "#FFFF00", "user.emerg": "#FFCCCB", "user.notice": "#00FFFF", "user.debug": "#008000", "user.crit": "#FF00FF" } } } ] }