goflow2 icon indicating copy to clipboard operation
goflow2 copied to clipboard
verified publisher icon netsampler

time_flow_start_ns and time_flow_end_ns is incorect value

Open majid-darvishfard opened this issue 1 year ago • 5 comments

hi i use GoFlow2 v2.1.3 (2024-03-24T04:51:36+0000) with this options : goflow2 -addr ":8081" -listen "netflow://10.104.32.2:5678/?count=2&workers=8&blocking=false&queue_size=100000" --format=json

time_flow_start_ns and time_flow_end_ns show eight hours later, but the value of time_received_ns is correct.

its sample output : "type":"NETFLOW_V9","time_received_ns":1713707202442970005,"time_flow_start_ns":1713677209000000000,"time_flow_end_ns":1713677209000000000

i check with netflow version 1 all time is correct

majid-darvishfard avatar Apr 21 '24 14:04 majid-darvishfard

Hi @majid-darvishfard, Thank you for reporting this issue Could you make sure your devices are on the correct timezone and on time? Time received is set by the collector. I won't be able to reproduce unless I dive into the samples

lspgn avatar Apr 21 '24 17:04 lspgn

this is output of cloudflare goflow v3.4.2 (2020-03-22T22:12:31+0000-travis)

Type:NETFLOW_V9 TimeReceived:1713794400 SequenceNum:866964284 SamplingRate:0 SamplerAddress:10.45.2.2 TimeFlowStart:1713794381 TimeFlowEnd:1713794382 Bytes:168 Packets:2 SrcAddr:188.212.69.241 DstAddr:5.106.12.7 Etype:2048 Proto:6 SrcPort:33994 DstPort:443 InIf:28 OutIf:20 SrcMac:00:00:00:00:00:00 DstMac:00:00:00:00:00:00 SrcVlan:0 DstVlan:41 VlanId:0 IngressVrfID:0 EgressVrfID:0 IPTos:0 ForwardingStatus:0 IPTTL:0 TCPFlags:16 IcmpType:0 IcmpCode:0 IPv6FlowLabel:0 FragmentId:0 FragmentOffset:0 BiFlowDirection:0 SrcAS:0 DstAS:0 NextHop:172.16.1.27 NextHopAS:0 SrcNet:32 DstNet:28 HasEncap:false SrcAddrEncap:<nil> DstAddrEncap:<nil> ProtoEncap:0 EtypeEncap:0 IPTosEncap:0 IPTTLEncap:0 IPv6FlowLabelEncap:0 FragmentIdEncap:0 FragmentOffsetEncap:0 HasMPLS:false MPLSCount:0 MPLS1TTL:0 MPLS1Label:0 MPLS2TTL:0, MPLS2Label: 0, MPLS3TTL:0 MPLS3Label:0 MPLSLastTTL:0 MPLSLastLabel:0 HasPPP:false PPPAddressControl:0
Type:NETFLOW_V9 TimeReceived:1713794400 SequenceNum:866964284 SamplingRate:0 SamplerAddress:10.45.2.2 TimeFlowStart:1713794372 TimeFlowEnd:1713794382 Bytes:2208 Packets:10 SrcAddr:151.235.27.60 DstAddr:188.114.97.6 Etype:2048 Proto:6 SrcPort:39228 DstPort:443 InIf:28 OutIf:20 SrcMac:00:00:00:00:00:00 DstMac:00:00:00:00:00:00 SrcVlan:0 DstVlan:41 VlanId:0 IngressVrfID:0 EgressVrfID:0 IPTos:0 ForwardingStatus:0 IPTTL:0 TCPFlags:16 IcmpType:0 IcmpCode:0 IPv6FlowLabel:0 FragmentId:0 FragmentOffset:0 BiFlowDirection:0 SrcAS:0 DstAS:0 NextHop:172.16.0.5 NextHopAS:0 SrcNet:32 DstNet:0 HasEncap:false SrcAddrEncap:<nil> DstAddrEncap:<nil> ProtoEncap:0 EtypeEncap:0 IPTosEncap:0 IPTTLEncap:0 IPv6FlowLabelEncap:0 FragmentIdEncap:0 FragmentOffsetEncap:0 HasMPLS:false MPLSCount:0 MPLS1TTL:0 MPLS1Label:0 MPLS2TTL:0, MPLS2Label: 0, MPLS3TTL:0 MPLS3Label:0 MPLSLastTTL:0 MPLSLastLabel:0 HasPPP:false PPPAddressControl:0
Type:NETFLOW_V9 TimeReceived:1713794400 SequenceNum:866964284 SamplingRate:0 SamplerAddress:10.45.2.2 TimeFlowStart:1713794381 TimeFlowEnd:1713794381 Bytes:78 Packets:1 SrcAddr:5.236.220.165 DstAddr:104.21.233.164 Etype:2048 Proto:6 SrcPort:14101 DstPort:443 InIf:28 OutIf:20 SrcMac:00:00:00:00:00:00 DstMac:00:00:00:00:00:00 SrcVlan:0 DstVlan:41 VlanId:0 IngressVrfID:0 EgressVrfID:0 IPTos:0 ForwardingStatus:0 IPTTL:0 TCPFlags:20 IcmpType:0 IcmpCode:0 IPv6FlowLabel:0 FragmentId:0 FragmentOffset:0 BiFlowDirection:0 SrcAS:0 DstAS:0 NextHop:172.16.0.5 NextHopAS:0 SrcNet:32 DstNet:0 HasEncap:false SrcAddrEncap:<nil> DstAddrEncap:<nil> ProtoEncap:0 EtypeEncap:0 IPTosEncap:0 IPTTLEncap:0 IPv6FlowLabelEncap:0 FragmentIdEncap:0 FragmentOffsetEncap:0 HasMPLS:false MPLSCount:0 MPLS1TTL:0 MPLS1Label:0 MPLS2TTL:0, MPLS2Label: 0, MPLS3TTL:0 MPLS3Label:0 MPLSLastTTL:0 MPLSLastLabel:0 HasPPP:false PPPAddressControl:0
Type:NETFLOW_V9 TimeReceived:1713794400 SequenceNum:866964284 SamplingRate:0 SamplerAddress:10.45.2.2 TimeFlowStart:1713794378 TimeFlowEnd:1713794383 Bytes:270 Packets:3 SrcAddr:5.236.109.220 DstAddr:142.250.186.42 Etype:2048 Proto:6 SrcPort:18386 DstPort:443 InIf:28 OutIf:20 SrcMac:00:00:00:00:00:00 DstMac:00:00:00:00:00:00 SrcVlan:0 DstVlan:41 VlanId:0 IngressVrfID:0 EgressVrfID:0 IPTos:0 ForwardingStatus:0 IPTTL:0 TCPFlags:16 IcmpType:0 IcmpCode:0 IPv6FlowLabel:0 FragmentId:0 FragmentOffset:0 BiFlowDirection:0 SrcAS:0 DstAS:0 NextHop:172.16.0.5 NextHopAS:0 SrcNet:32 DstNet:0 HasEncap:false SrcAddrEncap:<nil> DstAddrEncap:<nil> ProtoEncap:0 EtypeEncap:0 IPTosEncap:0 IPTTLEncap:0 IPv6FlowLabelEncap:0 FragmentIdEncap:0 FragmentOffsetEncap:0 HasMPLS:false MPLSCount:0 MPLS1TTL:0 MPLS1Label:0 MPLS2TTL:0, MPLS2Label: 0, MPLS3TTL:0 MPLS3Label:0 MPLSLastTTL:0 MPLSLastLabel:0 HasPPP:false PPPAddressControl:0

this is Goflow2 version v2.1.3 output

type=NETFLOW_V9 time_received_ns=1713794993921309890 sequence_num=2185666414 sampling_rate=0 sampler_address=10.143.143.254 time_flow_start_ns=1713765006000000000 time_flow_end_ns=1713765006000000000 bytes=64 packets=1 src_addr=5.219.133.20 dst_addr=142.250.184.202 etype=IPv4 proto=TCP src_port=23126 dst_port=443 in_if=0 out_if=66 src_mac=00:00:00:00:00:00 dst_mac=00:00:00:00:00:00 src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=0 ip_flags=0 tcp_flags=4 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=0 fragment_offset=0 src_as=0 dst_as=0 next_hop=0.0.0.0 next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop=0.0.0.0 bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=33025 observation_point_id=0
type=NETFLOW_V9 time_received_ns=1713794993921301031 sequence_num=2185666413 sampling_rate=0 sampler_address=10.143.143.254 time_flow_start_ns=1713765006000000000 time_flow_end_ns=1713765006000000000 bytes=591 packets=1 src_addr=100.78.118.71 dst_addr=172.217.18.3 etype=IPv4 proto=TCP src_port=33608 dst_port=443 in_if=0 out_if=66 src_mac=00:00:00:00:00:00 dst_mac=00:00:00:00:00:00 src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=0 ip_flags=0 tcp_flags=24 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=0 fragment_offset=0 src_as=0 dst_as=0 next_hop=0.0.0.0 next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop=0.0.0.0 bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=33025 observation_point_id=0
type=NETFLOW_V9 time_received_ns=1713794993916691555 sequence_num=2185666411 sampling_rate=0 sampler_address=10.143.143.254 time_flow_start_ns=1713763006000000000 time_flow_end_ns=1713763006000000000 bytes=180 packets=1 src_addr=37.254.194.32 dst_addr=172.66.46.247 etype=IPv4 proto=TCP src_port=37460 dst_port=443 in_if=0 out_if=66 src_mac=00:00:00:00:00:00 dst_mac=00:00:00:00:00:00 src_vlan=0 dst_vlan=0 vlan_id=0 ip_tos=0 forwarding_status=0 ip_ttl=0 ip_flags=0 tcp_flags=24 icmp_type=0 icmp_code=0 ipv6_flow_label=0 fragment_id=0 fragment_offset=0 src_as=0 dst_as=0 next_hop=0.0.0.0 next_hop_as=0 src_net=0.0.0.0/0 dst_net=0.0.0.0/0 bgp_next_hop=0.0.0.0 bgp_communities=[] as_path=[] mpls_ttl=[] mpls_label=[] mpls_ip=[] observation_domain_id=33025 observation_point_id=0

this is Goflow2 Version 1 output:

Type="NETFLOW_V9" TimeReceived=1713795774 SequenceNum=2186132033 SamplingRate=0 FlowDirection=1 SamplerAddress="10.143.143.254" TimeFlowStart=1713795737 TimeFlowEnd=1713795757 TimeFlowStartMs=1713795737000 TimeFlowEndMs=1713795757000 Bytes=148 Packets=2 SrcAddr="37.254.109.107" DstAddr="151.101.66.132" Etype=2048 Proto=6 SrcPort=11046 DstPort=80 InIf=0 OutIf=66 SrcMac="00:00:00:00:00:00" DstMac="00:00:00:00:00:00" SrcVlan=0 DstVlan=0 VlanId=0 IngressVrfId=0 EgressVrfId=0 IpTos=0 ForwardingStatus=0 IpTtl=0 TcpFlags=16 IcmpType=0 IcmpCode=0 Ipv6FlowLabel=0 FragmentId=0 FragmentOffset=0 BiFlowDirection=0 SrcAs=0 DstAs=0 NextHop="0.0.0.0" NextHopAs=0 SrcNet=0 DstNet=0 BgpNextHop=[0 0 0 0] BgpCommunities=[] AsPath=[] HasMpls=false MplsCount=0 Mpls_1Ttl=0 Mpls_1Label=0 Mpls_2Ttl=0 Mpls_2Label=0 Mpls_3Ttl=0 Mpls_3Label=0 MplsLastTtl=0 MplsLastLabel=0 MplsLabelIp=[] ObservationDomainId=0 ObservationPointId=0 CustomInteger_1=0 CustomInteger_2=0 CustomInteger_3=0 CustomInteger_4=0 CustomInteger_5=0 CustomBytes_1=[] CustomBytes_2=[] CustomBytes_3=[] CustomBytes_4=[] CustomBytes_5=[]
Type="NETFLOW_V9" TimeReceived=1713795774 SequenceNum=2186132033 SamplingRate=0 FlowDirection=1 SamplerAddress="10.143.143.254" TimeFlowStart=1713795757 TimeFlowEnd=1713795757 TimeFlowStartMs=1713795757000 TimeFlowEndMs=1713795757000 Bytes=295 Packets=1 SrcAddr="5.219.202.43" DstAddr="157.240.0.13" Etype=2048 Proto=6 SrcPort=42516 DstPort=443 InIf=0 OutIf=66 SrcMac="00:00:00:00:00:00" DstMac="00:00:00:00:00:00" SrcVlan=0 DstVlan=0 VlanId=0 IngressVrfId=0 EgressVrfId=0 IpTos=0 ForwardingStatus=0 IpTtl=0 TcpFlags=24 IcmpType=0 IcmpCode=0 Ipv6FlowLabel=0 FragmentId=0 FragmentOffset=0 BiFlowDirection=0 SrcAs=0 DstAs=0 NextHop="0.0.0.0" NextHopAs=0 SrcNet=0 DstNet=0 BgpNextHop=[0 0 0 0] BgpCommunities=[] AsPath=[] HasMpls=false MplsCount=0 Mpls_1Ttl=0 Mpls_1Label=0 Mpls_2Ttl=0 Mpls_2Label=0 Mpls_3Ttl=0 Mpls_3Label=0 MplsLastTtl=0 MplsLastLabel=0 MplsLabelIp=[] ObservationDomainId=0 ObservationPointId=0 CustomInteger_1=0 CustomInteger_2=0 CustomInteger_3=0 CustomInteger_4=0 CustomInteger_5=0 CustomBytes_1=[] CustomBytes_2=[] CustomBytes_3=[] CustomBytes_4=[] CustomBytes_5=[]
Type="NETFLOW_V9" TimeReceived=1713795774 SequenceNum=2186132033 SamplingRate=0 FlowDirection=1 SamplerAddress="10.143.143.254" TimeFlowStart=1713795757 TimeFlowEnd=1713795757 TimeFlowStartMs=1713795757000 TimeFlowEndMs=1713795757000 Bytes=154 Packets=1 SrcAddr="5.53.48.252" DstAddr="8.2.110.70" Etype=2048 Proto=6 SrcPort=24088 DstPort=443 InIf=0 OutIf=66 SrcMac="00:00:00:00:00:00" DstMac="00:00:00:00:00:00" SrcVlan=0 DstVlan=0 VlanId=0 IngressVrfId=0 EgressVrfId=0 IpTos=0 ForwardingStatus=0 IpTtl=0 TcpFlags=24 IcmpType=0 IcmpCode=0 Ipv6FlowLabel=0 FragmentId=0 FragmentOffset=0 BiFlowDirection=0 SrcAs=0 DstAs=0 NextHop="0.0.0.0" NextHopAs=0 SrcNet=0 DstNet=0 BgpNextHop=[0 0 0 0] BgpCommunities=[] AsPath=[] HasMpls=false MplsCount=0 Mpls_1Ttl=0 Mpls_1Label=0 Mpls_2Ttl=0 Mpls_2Label=0 Mpls_3Ttl=0 Mpls_3Label=0 MplsLastTtl=0 MplsLastLabel=0 MplsLabelIp=[] ObservationDomainId=0 ObservationPointId=0 CustomInteger_1=0 CustomInteger_2=0 CustomInteger_3=0 CustomInteger_4=0 CustomInteger_5=0 CustomBytes_1=[] CustomBytes_2=[] CustomBytes_3=[] CustomBytes_4=[] CustomBytes_5=[]

I test that on 3 different servers and on 3 different netflow device ; Goflow 2 version 1 and Cloudflare Goflow is okey . goflow2 version 2 start and end time is 8 hours ago.If the time zone is not correct, the other two versions should display the wrong time

what is differ between goflow2 version2 and goflow version 1 time calculated? why goflow version 2 time is Nanosecond?

majid-darvishfard avatar Apr 22 '24 14:04 majid-darvishfard

Thank you for this information, the comparison is great. Also noticing start_time=end_time on those packets..

This may be the same as https://github.com/netsampler/goflow2/issues/305 (I fixed for NetFlow v5 a similar issue https://github.com/netsampler/goflow2/issues/303) Are you able to send me a packet capture?

lspgn avatar Apr 22 '24 15:04 lspgn

Thank you for this information, the comparison is great. Also noticing start_time=end_time on those packets..

This may be the same as #305 (I fixed for NetFlow v5 a similar issue #303) Are you able to send me a packet capture?

It's sample pcap file test.zip

Let me know if you find a solution for this

majid-darvishfard avatar Apr 23 '24 11:04 majid-darvishfard

Thank you for this information, the comparison is great. Also noticing start_time=end_time on those packets..

This may be the same as #305 (I fixed for NetFlow v5 a similar issue #303) Are you able to send me a packet capture?

Hi Louis Did you check the sample file I sent?

If you need more file samples, send me your email

majid-darvishfard avatar Apr 29 '24 14:04 majid-darvishfard

Thank you The pcap does not contain any template unfortunately.

But could you have a look if https://github.com/netsampler/goflow2/pull/325 resolves your issue?

lspgn avatar May 19 '24 20:05 lspgn

hi @lspgn
I tested #325 and it was ok, the time problem is solved

majid-darvishfard avatar May 20 '24 14:05 majid-darvishfard