SamplingRate is zero in some devices

[amichalu]

In Mikrotik hEX PoE router SamplingRate is 0 so new dashboards are not showing traffic. In this case SamplingRate is simple 1.

[lspgn]

Hey @amichalu, Thank you for the report. Can you confirm that you/Mikrotik is using NetFlow v9?

[amichalu]

Yes, I confirm.

[lspgn]

Is there anything that references Option Templates anywhere? Can you configure manually the sampling? Will look into adding a CLI option in GoFlow that defines a fixed Sampling Rate.

In the meantime, I suggest you set the sampling at query time (are you using Clickhouse/Postgres or other?).

[amichalu]

Yes, thanks exactly what I did, added in query.

Src ip and dst ip addresses presented on grafana NetFlow Clickhouse dashboard are all from public range, none of them from my private network, is it ok ? Netflow is set on ether interface with NAT.

W dniu pon., 6.04.2020 o 00:24 lspgn [email protected] napisał(a):

Is there anything that references Option Templates anywhere? Can you configure manually the sampling? Will look into adding a CLI option in GoFlow https://github.com/cloudflare/goflow that defines a fixed Sampling Rate.

In the meantime, I suggest you set the sampling at query time (are you using Clickhouse/Postgres or other?).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cloudflare/flow-pipeline/issues/12#issuecomment-609494607, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARFBKFO5LPEJCOTG4E5TRDRLEAJZANCNFSM4L2HLELA .

-- Pozdrawiam, Adam Michalunio

[lspgn]

I'm assuming it's expected but I don't know enough how Mikrotik samples (actioned before DNAT on ingress and after SNAT on egress). Could you check with a packet capture?

[NiXuB86]

I have the same problem with my Mikrotik hap ac2 and use IPFIX. SamplingRate is 0 for every sample. Also there is problem with SQL queries i think(I'm not sure) because IPv4 addresses is inverted(Example: 3.25.168.192 instead of 192.168.25.3), because of that you can think that there is no private IP's. I intercepted traffic on docker bridge and in netflow packets IPv4 addresses is fine.

[amichalu]

Thanks a lot, I have not noticed inverted addresses !

[NiXuB86]

@amichalu Do you know the best way how to invert it back?

[amichalu]

So does it look that inverted addresses regard only Mikrotik ? Dont know how to deal with it now.

[NiXuB86]

No, that not regard only mikrotik, because as i said before, i captured traffic on docker bridge, and in IPFIX packets IP addresses is fine.

[NiXuB86]

I just tried to use netflow v9 instead of IPFIX, no difference in ClickHouse i have only inverted addresses and normal in netflow packets. Also with this query i have c0a8:1903:: instead of something like that 101:a8c0:: And also IPv6 addresses are fine

[NiXuB86]

I finally found a problem! It's caused by this function in query reinterpretAsUInt32(), it converts data to UInt32 and little endian from big endian, and IPv4NumToString() function expects UInt32 big endian. Because of that address is inverted. I didn't find any function to convert from little endian to big endian in ClickHouse docs.

[NiXuB86]

So I finally fixed it with this query: Normal IPv4: IPv4NumToString(reinterpretAsUInt32(substring(reverse(SrcAddr), 13, 4))) Inverted: IPv4NumToString(reinterpretAsUInt32(reverse(SrcAddr)))

And now I have the same problem with SamplingRate=0

[amichalu]

@NiXuB86 Thanks for quick solution to inverted addresses !