nProbe icon indicating copy to clipboard operation
nProbe copied to clipboard

Different sampling rate to each collector

Open shiv1997narayan opened this issue 6 years ago • 2 comments

Probe may send data to multiple collector and sub-collector Sampling rate to each collector could be separate.

shiv1997narayan avatar Apr 22 '19 10:04 shiv1997narayan

TESTCASE : Sending to different collector wih different sample rate

Observation ; Duration of obsevation is about 1 minute. with --sample-rate=1:50 , IPFIXCOL 3 get 11535 and IPFIXCOL 1 get 11536 messages with --sample-rate=1:50 and --sample-rate=1:75 , IPFIXCOL 3 get 16971 and IPFIXCOL 1 get 24111 messages with --sample-rate=1:25 and --sample-rate=1:50 , IPFIXCOL 3 get 39228 and IPFIXCOL 1 get 32494 messages with --sample-rate=1:75 and --sample-rate=1:1 , IPFIXCOL 3 get 47655 and IPFIXCOL 1 get 43353 messages

Increasing Sample-Rate should decrease the message , such trend is not show by the observation.

TESTCASE : Sending to kafka and IPFIXCOL simultaneously with different sample-rate

Observation: Messages in IPFIXCOL show variation according to --sample-rate but KAfka shows same no of messages .

For Duration 24seconds, with --sample-rate=1:50 , IPFIXCOL 1 get 10736 and kafka get 11579 messages with --sample-rate=1:75 , IPFIXCOL 1 get 7836 and kafka get 9958 messages with --sample-rate=1:50 and --sample-rate=1:75 , IPFIXCOL 1 get 9671 and kafka get 9841 messages with --sample-rate=1:75 and --sample-rate=1:50 , IPFIXCOL 1 get 13769 and kafka get 6859 messages

IPFIXCOL and Kafka is getting the data but as per observation we can assume that sample-rate that is taken is the same in different scenerio, So no different sample rate can be given in such case as well.

shiv1997narayan avatar May 16 '19 08:05 shiv1997narayan

Cento.conf file is as following in case for sending data to IPFIXCOL 192.168.144.52 and IPFIXCOL 192.168.144.72 :

--pid-file=/var/run/cento.pid -I=192.168.144.52:4740 --sample-rate=1:50 -I=192.168.144.72:4742 --sample-rate=1:75 --lifetime-timeout=600 --trace-log=/var/log/cento/cento.log #--kafka=192.168.144.63:9092,192.168.144.73:9092,192.168.144.83:9092,192.168.144.93:9092,192.168.144.53:9092;ipfix320;0;gzip #--sample-rate=1:25 --send-dont-wait --hash-size=1024000 --max-hash-size=2048000

Decode tunneled packets

--tunnel

Skip fragmented packets to increase processing speed

--skip-fragments
--max-socket-tx-buffer=1024MB

Specify the number of flows packets that are exported before flow templates are exported. Default 100.

--template-send-pkts=20000
--uniflows --dpi-level=2 --interface=nt:stream[0-19] --human-readable-tcpflags

To check the number of messages, sudo tcpdump host <Cento IP> is performed on IPFIXCOL and KAFKA node.

shiv1997narayan avatar May 16 '19 10:05 shiv1997narayan