Removing duplicated packets

[vpax]

I realize this might not be feasible resource-wise, but here's the wish: often in large environments, packet capture winds up with multiple copies of packets, for example from inside of different VLANs. They might not be quite precise L3 duplicates - in particular, the IP TTL may differ - but otherwise they're semantically redundant. It would be highly handy if Zeek provided a feature that would discard the later duplicates. Where this is germane, the window of time over which duplicates might appear is small (sub-msec), which helps with state-holding and not suppressing retransmissions.

One could also imagine Zeek learning whether its feed is subject to this problem and, if not, turning off the suppression (or, more generally, providing script-level events about the state of de-dup, along with BiFs for controlling it).