beats icon indicating copy to clipboard operation
beats copied to clipboard

[Filebeat 7.5.0 module cisco] host.name should not be the "Filebeat Agent Name"

Open MarcusCaepio opened this issue 4 years ago • 11 comments

Hi all, when you are using the cisco module, the host.name value is always the name of the "log collector". E.g. my cisco devices are 1.1.1.1, 1.1.2.1 and my filebeat runs on 1.1.1.2 the host.name value is always 1.1.1.2, which doesn't make sense, when using the SIEM map. Instead of this, it should be the address given in log.source.address, as this is the device, where the log comes from. I wrote an processor to replace the host.name with the correct ip, but it would be nice, to have this implemented in the future releases.

Regards, Marcus

MarcusCaepio avatar Dec 04 '19 13:12 MarcusCaepio

Hi Marcus, I am starting to use the CISCO module as well, mainly for IOS devices. What I would like to have is the dns name of the devices in addition or instead of the ip-address. So far I did not find a setting for this. Do you know if this can be done by the beat itself or do I need to implement it on my own?

Regards Bernhard

BFLB avatar Dec 05 '19 14:12 BFLB

Hi @BFLB I wrote a processor in the filebeat.yml:

processors:
  - drop_fields:
      when:
        equals:
          event.module: "cisco"
      fields: ["agent.ephemeral_id","agent.hostname","agent.id","agent.type","agent.version","host.name"]
  - dissect:
      when:
        equals:
          event.module: "cisco"
      tokenizer: "%{address}:%{}"
      field: "log.source.address"
      target_prefix: "host"
  - dns:
      when:
        equals:
          event.module: "cisco"
      type: reverse
      fields:
        host.address: host.name

drop fields: delete fields, which are unnecessary (imho) dissect: split the log.source.address, which is shown as "ip:port" into an host.address dns: resolve host.address and save it in host.name

As I am still testing this whole processor part, I don't know, if this is the elegant or the dirty way :D

Cheers, Marcus

MarcusCaepio avatar Dec 05 '19 15:12 MarcusCaepio

Hi @MarcusCaepio, Thank you for your fast response. As far as I can see, this looks good.

By the way, what devices are you using? I am asking since I am collecting logs of catalyst switches (IOS). So far it is working but the only log lines which are really parsed into fields are of access-lists (IOS Router/Firewall). Thus I plan to implement parsing of the most common catalyst logs like lineproto and link. Just to avoid duplicate work, please let me know if you are working in the same area.

Regards Bernhard

BFLB avatar Dec 05 '19 16:12 BFLB

You're welcome. Until now I just tested ASA logs. ^^

Cheers, Marcus

MarcusCaepio avatar Dec 05 '19 16:12 MarcusCaepio

Ok Cheers, Bernhard

BFLB avatar Dec 06 '19 07:12 BFLB

This problem seems the same on all Beats, the host.ip field is not calculated correctly when you use a Log Forwarder. Major issue: https://github.com/elastic/beats/issues/13920

chicco27 avatar Dec 27 '19 15:12 chicco27

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

botelastic[bot] avatar Feb 04 '21 17:02 botelastic[bot]

Hi,

Any news about this. I have the same in filebeat 7.10.2 with ios input:

  • module: cisco ios: enabled: true var.input: syslog var.syslog_host: 0.0.0.0 var.syslog_port: 9002
log.original | <190>443134: sw-lab-10: Feb 10 12:07:12.878: %HA_EM-6-LOG: catchall: disable
-- | --

But that is the only field where the switchname is..

-- Regards Falk

falkowich avatar Feb 10 '21 11:02 falkowich

Hi @BFLB I wrote a processor in the filebeat.yml:

processors:
  - drop_fields:
      when:
        equals:
          event.module: "cisco"
      fields: ["agent.ephemeral_id","agent.hostname","agent.id","agent.type","agent.version","host.name"]
  - dissect:
      when:
        equals:
          event.module: "cisco"
      tokenizer: "%{address}:%{}"
      field: "log.source.address"
      target_prefix: "host"
  - dns:
      when:
        equals:
          event.module: "cisco"
      type: reverse
      fields:
        host.address: host.name

drop fields: delete fields, which are unnecessary (imho) dissect: split the log.source.address, which is shown as "ip:port" into an host.address dns: resolve host.address and save it in host.name

As I am still testing this whole processor part, I don't know, if this is the elegant or the dirty way :D

Cheers, Marcus

Thank you Marcus,

This was just what I was looking for. I was struggling with the cisco.ios module, because of the way the logs were being listed. The only way I could search it was by looking to the "log.source.address" which still was a pain in the ass. Instead of giving an straight IP Address, it always returns with the IP Address with a random port attached (X.X.X.X:1234). Now it's way easier to search for each device.

Thank you once again.

rubenmachado avatar Jun 16 '21 15:06 rubenmachado

Hi Marcus, This was very helpful, thank you.

I have added the processor you suggested above, it worked well. Now in Kibana it shows the host.address as 'unknown field' . Guess this means I need to the field to the index next?

porebar avatar Oct 01 '21 15:10 porebar

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

botelastic[bot] avatar Oct 01 '22 16:10 botelastic[bot]