why do octetDeltaCount and packetDeltaCount show up as hex strings in Netflow v9 and integers in IPFIX?

[jgc234]

More of a questions than issue.. In the JSON output for Netflow v9, the fields for octetDeltaCount(1) and packetDeltaCount(2) show up as hex strings, whereas in IPFIX they're integers. Is this supposed to be the case?

The marhsal code for v9 uses the same element definitions from ipfix/rfc5102_model.go .. or is it related to how the template defines these values (the RFC says they're nominal length is 4 bytes, which is different to the firm definition of unsigned64 in IPFIX). In the end, these fields have a type of []uint8.

[lwsbox]

puzzled too

[lwsbox]

In interpret.go, the octetDeltaCount is 4 bytes which is less than t.minLen(), so set this filed with []uint8. after changing the unsigned64 to unsigned32, the filed do not show up as hex strings.

logstash-netflow plugin use two template config to distinguish netflow V9 and ipfix, maybe vflow should use two model.go too?

// Interpret read data fields based on the type - big endian
func Interpret(b *[]byte, t FieldType) interface{} {
	if len(*b) < t.minLen() {
		return *b
	}

[eksantrik]

Hello folks,

We are trying to determine what fields in Netflow v9 are reported in Hex. Do you know of a resource on the internet that gives the exact field types for Netflow v9?

If this is an issue due to how vflow is processing the incoming data, do you know if it has been fixed yet?

Thanks

[dbardbar]

@eksantrik , @lwsbox, @jgc234

I was also very puzzled by this. After some digging, I can say this is not a v9 vs ipfix issue. The fields in nf9 and ipfix define a standard length, like unsigned64, unsigned8, etc. See https://www.iana.org/assignments/ipfix/ipfix.xhtml

In vflow the code has a copy of that table, in rfc5102_model.go.

However, it seems that many implementations out there, and I don't think this is a v9 vs. ipfix thing, but more of a per-vendor decision, they decide to send some fields with sizes smaller than the size declared for it in the RFC. I'm not sure if the RFC allows this or not, but there seems to be quite a few implementations that do it.

As @lwsbox wrote, there's a check in ipfix/decoder.go and netflow/v9/decoder.go which reads bytes from the packet based on the length that appeared in the template, but tries to interpret them as types defined in the RFC. If the length in the packet is smaller than the length specified in the RFC, then vflow will hold that field as a byte array, and that would serialized in the JSON as a hex string.

So yes, @eksantrik , this is a bug or at least a very annoying behaviour in vflow. If we compare this to Wireshark parser, it doesn't exhibit this odd behaviour.

The fix for this could be a bit tricky for the general case.

For unsigned8/16/32/64 types, the Interpret method could be modified to see if the size is smaller than expected, and if so create a new byte array with zero padding in the beginning, and put in the bytes read from the packet at the 'LSB' part of the new byte array. Need to be careful here with endianity.

For signed8/16/32/64 - not sure what to do regarding the most significant bit which indicates sign, but those seem to be rare.

For special types like IPs, MACs, etc - I've yet to see any pcap with this behaviour for them.

I'll perhaps invest some time to create a fix for this, as I agree this is super-annoying, and causes me grief and more work on my code that reads vflow's output.

WDYT?