Performance Question

[Gabscap]

Hello,

I'm using goflow2 to write incoming IPFIX into Kafka/Protobuf. After running goflow for some time, it starts to buffer incoming packets and can't keep up with the load. All this while only using roughly one core (100-125%). Is there any way for goflow to utilize more than a single core?

cmdline: goflow2 -addr :18702 -transport=kafka -transport.kafka.version=4.0.0 -transport.kafka.brokers=192.168.0.1:9092 -transport.kafka.topic=flows -format=bin -transport.kafka.compression=lz4 "-listen=netflow://:8702/?count=4&workers=64&queue_size=1000000&blocking=false"

Here some metrics:

[lspgn]

Hello, Is the quote before -listen supposed to be there? Are you using GOMAXPROCS somewhere?

[Gabscap]

The matching quote is at the end of the cmdline. It's there, because & has a special meaning in bash. No GOMAXPROCS is not used. I tried setting it explicitly to 32, but it did not change anything.

[Nachtfalkeaw]

@Gabscap i think you config or cmd IS wrong or Not working as expected.

I Used default config and Goflow2 was limited by CPU. I Changed worker and it was working as expected. So maybe co dig IS wrong or limitation is based e.g. in Kafka. You May try to write to File and See If it Changes Something.

From the Docs there are several possibilities to Set worker:

$ goflow2 -listen flow://:6343/?count=4&workers=16&blocking=false&queue_size=1000000 ^ ^ ^ ^ ┃ ┃ ┃ ┗ In buffered mode, the amount of packets stored in memory ┃ ┃ ┗ Real-time mode ┃ ┗ Decoding workers ┗ Open sockets listening

Or this: $ ./goflow2 -listen 'sflow://:6343?count=4,netflow://:2055'

Maybe try with only "worker" and without "Count" to avoid the & I Addition you can try to avoid blocking and Queue size to avoid additional & If you think this is the issue.

Unfortunately i do Not have Access to my config right now to verify.

[Gabscap]

This problem was solved by running GoFlow2 v2.2.3 (2025-06-04T04:14:06+0000)