Process ipfix: Sequencer run - resize output buffer

[Gabscap]

Hello,

I am running some nfcapd processes. After some time the nfcapd crashes by spamming Process ipfix: Sequencer run - resize output buffer over and over again. What does this error mean?

I'm using the release version 1.7.6 with following command line:

nfcapd -S 1 -z=lz4:10 -W 8 -B 1024000 -w /mnt/nfcapd/somedir -p 1234 -t 60 -e -v

Incoming flows originate from yaf+nfacctd with tee plugin.

[phaag]

Would it be possible to collect the pcap data sent to the collector until it crashes? If so could you make me available that pcap? You can send me the derails to my email address in the AUTHORS file.

What are the version und cmd line option of yaf/nfacctd?

[Gabscap]

Unfortunately I can't share the data because of data protection laws. I'm running multiple instances of nfcapd for different flow meters across the network. This error only happens on busy measurement points(2-6 Gbit/s).

Maybe the DPI_PLUGIN of yaf is causing the problem. Yesterday, I disabled the DPI_PLUGIN on a yaf instance and its nfcapd process is still running.

yaf

yaf --config /usr/local/etc/yaf.init --no-vlan-in-key

yaf.init:

input = {inf="enp101s0f1np1", type="pfring"}

output = {host="127.0.0.1", port="18000", protocol="udp", udp_temp_timeout=60}

log = {spec="/var/log/yaf.log", level="debug"}

DPI_PLUGIN = {
    -- The "name" keyword specifies the full path to the plugin
    -- library name to load.
    name = "/usr/local/lib/yaf/dpacketplugin.la",

    options = "53",

    -- The "conf" keyword specifies the path to a configuration
    -- file to be given to the plugin.
    conf = "/usr/local/etc/yafDPIRules.conf"
}

plugin = { DPI_PLUGIN }

applabel = true
applabel_rules = "/usr/local/etc/yafApplabelRules.conf"
maxpayload = 384

stats = 300

export = {
    silk = false,
    uniflow = true
}

idle_timeout = 60
active_timeout = 60

nfacctd

nfacctd -f /usr/local/etc/nfacctd.conf

nfacctd.conf:

nfacctd_port: 18000
nfacctd_ip: 127.0.0.1
!
plugins: tee[repl]
tee_receivers[repl]: /usr/local/etc/nfacctd_receivers.conf
tee_transparent: false
!
! pre_tag_map: /path/to/pretag.map
!
plugin_buffer_size: 10240
plugin_pipe_size: 1024000
nfacctd_pipe_size: 1024000

[phaag]

Thanks! - Yes - it could be the DPI_PLUGIN - I will certainly check that.

I possible, could you compile the nfdump tools again and replace -O3 with -fsanitize=address -O1 Then run again with all plugins enabled. If nfcapd crashes, the sanatizer will spit out some debug and adress information. If you could post that, it will help to debug.

Many thanks

[Gabscap]

But nfcapd does not really crash. It goes into an endless loop of writing Process ipfix: Sequencer run - resize output buffer over and over again. Does it still work in this case?

[phaag]

Ok - then leave it for the moment. I will habe time to debug that on the weekend.

[phaag]

Which yaf and nfacctd versions are you using? Which OS/version CPU arch?

[phaag]

I had no luck.. yaf crashes on my side. I changed the ipfix code to be less chatty when a buffer resize occurs. That may happen, if a var length payload field is processed.

[phaag]

If you manage to collect a pcap from an uncritial environment, from a lab or so without personal data, I'd happily continue on this case. Another option would be, if you have a VM you could share with your setup tools etc, which I could do some test.

[Gabscap]

Version information:

$ yaf -V
yaf version 2.16.2  Build Configuration:
    * Timezone support:                 UTC
    * Fixbuf version:                   2.5.1
    * DAG support:                      NO
    * Napatech support:                 NO
    * Netronome support:                NO
    * Bivio support:                    NO
    * PFRING support:                   YES
    * Compact IPv4 support:             YES
    * Plugin support:                   YES
    * Application Labeling:             YES
    * Payload Processing Support:       YES
    * Entropy support:                  NO
    * Fingerprint Export Support:       NO
    * P0F Support:                      NO
    * Spread Support:                   NO
    * MPLS Support:                     NO
    * Non-IP Support:                   NO
    * Separate Interface Support:       YES
    * nDPI Support:                     NO
    * IE Metadata Export:               YES
 (c) 2006-2024 Carnegie Mellon University.
GNU General Public License (GPL) Rights pursuant to Version 2, June 1991
Some included library code covered by LGPL 2.1; see source for details.
Send bug reports, feature requests, and comments to [email protected].

$ nfacctd -V
NetFlow Accounting Daemon, nfacctd 1.7.9 [RELEASE]

Arguments:
 '--prefix=/usr/local' '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'

Libs:
cdada 0.6.0
libpcap version 1.10.1 (with TPACKET_V3)

Plugins:
memory
print
nfprobe
sfprobe
tee

System:
Linux 6.8.0-60-generic #63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 19:04:15 UTC 2025 x86_64

Compiler:
gcc 13.3.0

For suggestions, critics, bugs, contact me: Paolo Lucente <[email protected]>.

I'll try to generate some data for you to reproduce this issue.

[phaag]

As I cannot reproduce this error and due to lack of further data, I close this issue. In case of more information and data is available, feel free to reopen the issue.