ntop
nDPI payload length checks and MTU
Some protocol dissectors (especially the TCP ones) are using length checks from dissected payload and comparing it with payload_packet_len to successfully detect (or not) a certain protocol. As it is effective, because there are low chances for false-positives within other protocols, it may also dependent on the MTU size of the network interface.
For example: The ElasticSearch dissector uses that detection pattern. But a single ES message can easily exceed the MTU on the network interface nDPI is running on, because it is split into smaller fragments that do not exceed the MTU, causing a false-negative detection.
As by now nDPI does not provide any support to reassemble TCP fragments. But IMHO we could at least extend nDPI to provide two new API functions e.g.:
ndpi_set_mtu(ndpi_struct, mtu); // set MTU allowed for this nDPI instance
ndpi_check_packet_length(ndpi_struct, packet, dissected_length); // returns 1 iff packet->payload_packet_len matches dissected_length OR dissected_length is greater than MTU, 0 otherwise
Downside: This may increase false-positive rate. Does this make sense? What do you guys think?
(can not share the ES PCAP as it contains data I do not own)
- I think it's useful to have such apis as i had a similar experience with a private protocol
- some concerns:
- how to set a reliable value for MTU ( not so sure if the MTU of capture interface is enough)
- should only used by a specific dissector (dissectors where we know what we can do with next data chunk)
- what MAX dissected len should be allowed as it could be corrupted (maybe cap for jumbo frame 9k)
