secdev
Netflow field name with spaces cause error
Brief description
During netflow packet generation got error if template have fields names with spaces. Python module inspect checks failed because this fields is not idendifier-compliant
https://docs.python.org/3.10/reference/lexical_analysis.html#identifiers
Probably, this check not only netflow-field related.
Scapy version
2.5.0
Python version
3.10
Operating system
Linux 5.15.0
Additional environment information
No response
How to reproduce
from scapy.all import *
flowset = NetflowFlowsetV9(
templates=[NetflowTemplateV9(
template_fields=[
NetflowTemplateFieldV9(fieldType="IPV4_SRC_ADDR", fieldLength=4),
NetflowTemplateFieldV9(fieldType="IPV4_DST_ADDR", fieldLength=4),
NetflowTemplateFieldV9(fieldType="IPV4_NEXT_HOP", fieldLength=4),
NetflowTemplateFieldV9(fieldType="IN_PKTS", fieldLength=8),
NetflowTemplateFieldV9(fieldType="IN_BYTES", fieldLength=8),
NetflowTemplateFieldV9(fieldType="IP TTL MINIMUM", fieldLength=1),
NetflowTemplateFieldV9(fieldType="IP TTL MAXIMUM", fieldLength=1),
NetflowTemplateFieldV9(fieldType="PROTOCOL", fieldLength=1),
],
templateID=1510,
fieldCount=8)
],
flowSetID=0
)
recordClass = GetNetflowRecordV9(flowset)
header = Ether()/IP()/UDP()
netflow_header = NetflowHeader()/NetflowHeaderV10()
pkt = header / netflow_header / flowset
pkt.show()
Actual result
Netflow packet generate failed with fields with spaces in names like "IP TTL MINIMUM" "IP TTL MAXIMUM"
Traceback (most recent call last):
File "/home/user/ipfix/nfv10.py", line 22, in <module>
recordClass = GetNetflowRecordV9(flowset)
File "/home/user/.local/lib/python3.10/site-packages/scapy/layers/netflow.py", line 1384, in GetNetflowRecordV9
cls = _GenNetflowRecordV9(NetflowRecordV9, llist)
File "/home/user/.local/lib/python3.10/site-packages/scapy/layers/netflow.py", line 1364, in _GenNetflowRecordV9
class NetflowRecordV9I(cls):
File "/home/user/.local/lib/python3.10/site-packages/scapy/base_classes.py", line 341, in __new__
] + [
File "/home/user/.local/lib/python3.10/site-packages/scapy/base_classes.py", line 342, in <listcomp>
inspect.Parameter(f.name,
File "/usr/lib/python3.10/inspect.py", line 2666, in __init__
raise ValueError('{!r} is not a valid parameter name'.format(name))
ValueError: 'IP TTL MINIMUM' is not a valid parameter name
Expected result
Get nf packet with full template set, if I drop problem fields, inspect checks comlete without problem:
###[ Ethernet ]###
dst = ff:ff:ff:ff:ff:ff
src = 00:00:00:00:00:00
type = IPv4
###[ IP ]###
version = 4
ihl = None
tos = 0x0
len = None
id = 1
flags =
frag = 0
ttl = 64
proto = udp
chksum = None
src = 127.0.0.1
dst = 127.0.0.1
\options \
###[ UDP ]###
sport = 2055
dport = 2055
len = None
chksum = None
###[ Netflow Header ]###
version = 10
###[ IPFix (Netflow V10) Header ]###
length = None
ExportTime= Thu, 01 Jan 1970 00:00:00 +0000 (0)
flowSequence= 0
ObservationDomainID= 0
###[ Netflow FlowSet V9/10 ]###
flowSetID = 0
length = None
\templates \
|###[ Netflow Flowset Template V9/10 ]###
| templateID= 1510
| fieldCount= 8
| \template_fields\
| |###[ Netflow Flowset Template Field V9/10 ]###
| | enterpriseBit= 0
| | fieldType = IPV4_SRC_ADDR
| | fieldLength= 4
| |###[ Netflow Flowset Template Field V9/10 ]###
| | enterpriseBit= 0
| | fieldType = IPV4_DST_ADDR
| | fieldLength= 4
| |###[ Netflow Flowset Template Field V9/10 ]###
| | enterpriseBit= 0
| | fieldType = IPV4_NEXT_HOP
| | fieldLength= 4
| |###[ Netflow Flowset Template Field V9/10 ]###
| | enterpriseBit= 0
| | fieldType = IN_PKTS
| | fieldLength= 8
| |###[ Netflow Flowset Template Field V9/10 ]###
| | enterpriseBit= 0
| | fieldType = IN_BYTES
| | fieldLength= 8
| |###[ Netflow Flowset Template Field V9/10 ]###
| | enterpriseBit= 0
| | fieldType = PROTOCOL
| | fieldLength= 1
Related resources
https://github.com/secdev/scapy/blob/master/scapy/layers/netflow.py#L251
Couldn't you use IP_TTL_MINIMUM. It's a scapy limitation to not support spaces in fields.
Yes, if I change spaces to underscores in filed-names https://github.com/secdev/scapy/blob/master/scapy/layers/netflow.py#L251 problem solved. Not certainly sure where this dictionary (with codes to names mapping) came from, but looks like it is common with one used in other tools like wireshark.
It might be worth considering changing the dictionary to be fully comply with scapy's limitations.
The dictionary has indeed a reference to the one used in wireshark.
In both versions, it is indeed a mix of original Netflow v9 fields in CAPITAL_CASE and ipfix fields in camelCase. See the comparative here.
It might be valuable to switch all entries on the IANA IPFIX normalized names, as they ensure backward compatibility with Netflow.
Otherwise, I can just remove the spaces in the faulty Netflow V9 fields, according to Cisco they should be respectively MIN_TTL, MAX_TTL and IPV4_IDENT.
@theplatypus
I found two more fields in addition to the ones mentioned above
313: "IP_SECTION HEADER",
314: "IP_SECTION PAYLOAD",
Second option (removing spaces) more preferable, i think. The first one looks like break changes (i may be wrong).
That's a good finding! Thanks for sharing
@gpotter2 , what do you think about switching to IANA IPFIX field names for that dictionnary ? If you agree I can start working on it