(File)Beats failed to get udp stats from /proc issue - IPv4/IPv6 bug

[Daniel314]

Hi,

When using TCP and UDP inputs under Filebeats (version 8.12.0), sometimes the listeners are listening only under IPv4, and sometimes they're listening under IPV4/IPv6. For every listener on both IPv4/IPv6, Filebeat generates an error like the following every minute:

WARN [input.udp] map[file.line:251 file.name:udp/input.go function:github.com/elastic/beats/v7/filebeat/input/udp.(*inputMetrics).poll] failed to get udp stats from /proc: /proc/net/udp entry not found for [00000000:09D2] {"ecs.version": "1.6.0"}

This is for Filebeat 8.12.0, running under Ubuntu 22.04. This was raised on the Elastic discussion forum, but no one commented on it. Here is the forum post: https://discuss.elastic.co/t/elastic-agent-entry-proc-net-udp-not-found/348140

I know it's an IPv4/IPv6 listening issue because the port(s) that filebeats opens exclusively under IPv4 (e.g. /proc/net/udp), filebeats doesn't complain about. However, the ports that are opened under IPv6 (they are listed in /proc/net/udp6), filebeats reports the above error. This is true for both TCP and UDP listeners.

Here's the relevant section of my filebeats.yml file:

filebeat.inputs:

• type: tcp host: "0.0.0.0:1514" tags: [ "mycustomtag1" ] fields_under_root: true fields: MYtype: syslog

• type: udp host: "0.0.0.0:2514" fields_under_root: true fields: MYtype: wifi

• type: udp host: "localhost:3514" fields_under_root: true fields: MYtype: syslog

• type: udp host: "0.0.0.0:7401" fields_under_root: true fields: MYtype: device

For the above config, filebeats complains about UDP ports 2514 and 7401, and TCP port 1514, but not UDP port 3514. Looking at /proc/net/udp, it has an entry for port 3514 (0DBA), but not port 2514 or 7401. Looking at /proc/net/udp6, it has entries for port 2514 (09D2) and 7401 (1CE9). Looking at /proc/net/tcp, there is no entry for 1514. Looking at /proc/net/tcp6, there is an entry for 1514 (05EA).

Thanks,

 - Daniel

[botelastic[bot]]

This issue doesn't have a Team:<team> label.

[ar3diu]

I've noticed the same thing in Elastic Agent when using integrations for TCP/UDP inputs. Did you manage to find a fix?

[Daniel314]

The only workaround I can think of is to completely disable the IPv6 stack on the host, to force the listening connection to only be on the IPv4 interface (i.e. /proc/net/tcp). Something like this: (echo 'net.ipv6.conf.all.disable_ipv6 = 1'; echo 'net.ipv6.conf.default.disable_ipv6 = 1'; echo 'net.ipv6.conf.lo.disable_ipv6 = 1') > /etc/sysctl.d/99-noipv6.conf Then reboot the host.

[cdonahue55]

Someone added 99-noipv6.conf on April 25

root@02-saas-ap3-esb-151899-010:/etc/sysctl.d# ls -l total 12 -rw-r--r--. 1 root root 19 Jan 6 2023 50-fs.conf -rw-r--r--. 1 root root 74 Aug 29 2022 99-disable_ipv6.conf -rw-r--r--. 1 root root 108 Apr 25 23:19 99-noipv6.conf

This appears to be the solution and will be incorporated into the next template