pysnmp
pysnmp copied to clipboard
etingof
bulkCmd() returning varbind not in scope of requested OID when maxRepetitions == (numSubElements + 1)
we recently switched from easysnmp to pysnmp just because of the excellent MIB services. I'm quite happy with pysnmp, although we needed to fix at least 3 to 4 issues in various Cisco MIBs. But anyway, using the hlapi in conjunction with the MIB sevices is a great pleasure!
Having said that we came across an issue in a very particular situation: Calling cmdgen.CommandGenerator().bulkCmd(...) with maxRepititions set to a number that is exactly one above the number of elements to be returned from the last SNMP GETBULK response will return exaxtly one varbind to much. The surplus varbind contains the next OID not in the scope of the requested OID with an empty value. So, if you only check for the last element in the returned OID (the index) and not the whole OID you will most likely get an index of 1 and thus clear your first result when copying out the results.
Example: if you do a bulkCmd() for "IF-MIB::ifDescr" with maxRepititions set to 10 (or 5) and it happens that you should get 9 varbinds (interface descriptions) you will get 10 varbinds, the last being not in the scope of "IF-MIB::ifDescr".
Also, I observed in Wireshark that in this particular situation bulkCmd() will do one SNMP GETBULK too much.
I'm _attaching a python script to check this issue:
I could reproduce the issue on various Cisco device types and different OIDs. Here's the output of the test script for a device having 9 interfaces with results being OK ( numRepetitions = 4 or 8) and being not OK ( numRepetitions = 5 or 10):
----------- snip -------------------- ./getIfDescrs.py 172.16.31.185 public 4 Interface: 1 oid: 1.3.6.1.2.1.2.2.1.2.1, val: Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001 Interface: 2 oid: 1.3.6.1.2.1.2.2.1.2.2, val: Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001 Interface: 3 oid: 1.3.6.1.2.1.2.2.1.2.3, val: Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001 Interface: 4 oid: 1.3.6.1.2.1.2.2.1.2.4, val: Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001 Interface: 5 oid: 1.3.6.1.2.1.2.2.1.2.5, val: Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001 Interface: 6 oid: 1.3.6.1.2.1.2.2.1.2.6, val: Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001 Interface: 7 oid: 1.3.6.1.2.1.2.2.1.2.7, val: Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001 Interface: 8 oid: 1.3.6.1.2.1.2.2.1.2.8, val: Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001 Interface: 9 oid: 1.3.6.1.2.1.2.2.1.2.9, val: Virtual Interface
./getIfDescrs.py 172.16.31.185 public 5 Interface: 1 oid: 1.3.6.1.2.1.2.2.1.2.1, val: Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001 Interface: 2 oid: 1.3.6.1.2.1.2.2.1.2.2, val: Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001 Interface: 3 oid: 1.3.6.1.2.1.2.2.1.2.3, val: Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001 Interface: 4 oid: 1.3.6.1.2.1.2.2.1.2.4, val: Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001 Interface: 5 oid: 1.3.6.1.2.1.2.2.1.2.5, val: Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001 Interface: 6 oid: 1.3.6.1.2.1.2.2.1.2.6, val: Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001 Interface: 7 oid: 1.3.6.1.2.1.2.2.1.2.7, val: Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001 Interface: 8 oid: 1.3.6.1.2.1.2.2.1.2.8, val: Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001 Interface: 9 oid: 1.3.6.1.2.1.2.2.1.2.9, val: Virtual Interface Interface: 10 oid: 1.3.6.1.2.1.2.2.1.3.1, val:
./getIfDescrs.py 172.16.31.185 public 8 Interface: 1 oid: 1.3.6.1.2.1.2.2.1.2.1, val: Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001 Interface: 2 oid: 1.3.6.1.2.1.2.2.1.2.2, val: Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001 Interface: 3 oid: 1.3.6.1.2.1.2.2.1.2.3, val: Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001 Interface: 4 oid: 1.3.6.1.2.1.2.2.1.2.4, val: Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001 Interface: 5 oid: 1.3.6.1.2.1.2.2.1.2.5, val: Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001 Interface: 6 oid: 1.3.6.1.2.1.2.2.1.2.6, val: Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001 Interface: 7 oid: 1.3.6.1.2.1.2.2.1.2.7, val: Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001 Interface: 8 oid: 1.3.6.1.2.1.2.2.1.2.8, val: Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001 Interface: 9 oid: 1.3.6.1.2.1.2.2.1.2.9, val: Virtual Interface
./getIfDescrs.py 172.16.31.185 public 10 Interface: 1 oid: 1.3.6.1.2.1.2.2.1.2.1, val: Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001 Interface: 2 oid: 1.3.6.1.2.1.2.2.1.2.2, val: Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001 Interface: 3 oid: 1.3.6.1.2.1.2.2.1.2.3, val: Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001 Interface: 4 oid: 1.3.6.1.2.1.2.2.1.2.4, val: Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001 Interface: 5 oid: 1.3.6.1.2.1.2.2.1.2.5, val: Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001 Interface: 6 oid: 1.3.6.1.2.1.2.2.1.2.6, val: Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001 Interface: 7 oid: 1.3.6.1.2.1.2.2.1.2.7, val: Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001 Interface: 8 oid: 1.3.6.1.2.1.2.2.1.2.8, val: Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001 Interface: 9 oid: 1.3.6.1.2.1.2.2.1.2.9, val: Virtual Interface Interface: 10 oid: 1.3.6.1.2.1.2.2.1.3.1, val: ----------- snap --------------------
Thnx for the good work!
I think maxRepetitions parameter is not designed to cut by some OIDs conditionally - it just limits the maximum number of whatever OIDs the manager is willing to receive.
Have you considered using newer pysnmp API (e.g. hlapi) along with lexicographicMode=False parameter which should cut returned OIDs by columnar OID (for example).
Does it make sense?
Agree, maxRepetitions gives the max number of elements to be returned by SNMP GET BULK and this works fine. I was assuming that cmdGen.bulkCmd() will do the cut when the result elements are leaving the initial OID scope (which it does in most cases).
Anyway, I see that I was using an older API, so I recoded it in hlapi with lexicographicMode=False which describes exactly what I was trying to achieve.
Unfortunately, the result is mostly the same: if maxRepititions is set to a value so that the last GET BULK will return exactly one additional element outside the initial scope, then bulkCmd() will return this extra element with a value of No more variables left in this MIB View. In all other case, bulkCmd(...,lexicographicMode=False,..) behaves like expected, returning only the elements below the initial OID.
Here's the script output for maxRepitions = 4 (OK) and 5 (one too much):
./hlapiGetIfDescrs.py 172.16.31.185 public 4 IF-MIB::ifDescr.1 = Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001 IF-MIB::ifDescr.2 = Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001 IF-MIB::ifDescr.3 = Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001 IF-MIB::ifDescr.4 = Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001 IF-MIB::ifDescr.5 = Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001 IF-MIB::ifDescr.6 = Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001 IF-MIB::ifDescr.7 = Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001 IF-MIB::ifDescr.8 = Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001 IF-MIB::ifDescr.9 = Virtual Interface
./hlapiGetIfDescrs.py 172.16.31.185 public 5 IF-MIB::ifDescr.1 = Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001 IF-MIB::ifDescr.2 = Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001 IF-MIB::ifDescr.3 = Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001 IF-MIB::ifDescr.4 = Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001 IF-MIB::ifDescr.5 = Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001 IF-MIB::ifDescr.6 = Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001 IF-MIB::ifDescr.7 = Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001 IF-MIB::ifDescr.8 = Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001 IF-MIB::ifDescr.9 = Virtual Interface IF-MIB::ifType.1 = No more variables left in this MIB View
Alright, this OID leaking is not something that should happen. Let's see if commit e066870f59fc3918ec7dee88c3d1316e075f86ec (#172) fixes this issue? Just keep in mind that it's presently based on the pysnmp-4.4.5 branch.
Thanks!
Thanks! I'll give it a try when I'm back in the office beginning of August
Meanwhile, I have to merge this to get this fix released. Feel free to reopen this issue if it still does not work as it should. ;-)
Cheers!
With 4.4.5 it's different, but still not what I was expecting:
./hlapiGetIfDescrs.py 172.16.31.185 public 4
IF-MIB::ifDescr.1 = Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001
IF-MIB::ifDescr.2 = Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001
IF-MIB::ifDescr.3 = Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001
IF-MIB::ifDescr.4 = Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001
IF-MIB::ifDescr.5 = Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001
IF-MIB::ifDescr.6 = Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001
IF-MIB::ifDescr.7 = Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001
IF-MIB::ifDescr.8 = Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001
IF-MIB::ifDescr.9 = Virtual Interface
./hlapiGetIfDescrs.py 172.16.31.185 public 5
IF-MIB::ifDescr.1 = Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001
IF-MIB::ifDescr.2 = Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001
IF-MIB::ifDescr.3 = Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001
IF-MIB::ifDescr.4 = Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001
IF-MIB::ifDescr.5 = Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001
IF-MIB::ifDescr.6 = Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001
IF-MIB::ifDescr.7 = Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001
IF-MIB::ifDescr.8 = Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001
IF-MIB::ifDescr.9 = Virtual Interface
IF-MIB::ifDescr.9 = No more variables left in this MIB View
Now, the last OID is repeated with the value of 'No more variables left in this MIB View'. Unfortunately, it's now even harder to recognize the extra OID/value pair, because the OID is perfectly in the scope of the requested OID.
I would like to re-opened the issue but I can't do it.
because the OID is perfectly in the scope of the requested OID.
You mean we should not report that EOM oid-value at the end?
Trouble is that if you walk 1+ OIDs at the same time, the EOM condition for different OIDs can happen at different steps. In that case we are kind of forced to fill that EOM sentinel for the value of EOM OIDs for as long as we have at least one OID not reaching its EOM...
How would we best approach this?
I wasn't thinking about use case of walking different OIDs at the same time. I can see the reason for returning the EOM for 'shorter' OIDs. So, I tried it for two OIDs within different tables to see how this works:
./hlapiGetIfDescrs.py 172.16.31.185 public 4
IP-MIB::ipAdEntAddr.0.0.0.0 = 0.0.0.0
IF-MIB::ifDescr.1 = Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.1.1.1.1 = 1.1.1.1
IF-MIB::ifDescr.2 = Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.16.15.253 = 172.16.15.253
IF-MIB::ifDescr.3 = Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.16.31.106 = 172.16.31.106
IF-MIB::ifDescr.4 = Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.16.31.185 = 172.16.31.185
IF-MIB::ifDescr.5 = Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.24.15.253 = 172.24.15.253
IF-MIB::ifDescr.6 = Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.24.79.253 = 172.24.79.253
IF-MIB::ifDescr.7 = Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.24.79.253 = No more variables left in this MIB View
IF-MIB::ifDescr.8 = Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.24.79.253 = No more variables left in this MIB View
IF-MIB::ifDescr.9 = Virtual Interface
./hlapiGetIfDescrs.py 172.16.31.185 public 5
IP-MIB::ipAdEntAddr.0.0.0.0 = 0.0.0.0
IF-MIB::ifDescr.1 = Unit: 0 Slot: 0 Port: 1 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.1.1.1.1 = 1.1.1.1
IF-MIB::ifDescr.2 = Unit: 0 Slot: 0 Port: 2 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.16.15.253 = 172.16.15.253
IF-MIB::ifDescr.3 = Unit: 0 Slot: 0 Port: 3 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.16.31.106 = 172.16.31.106
IF-MIB::ifDescr.4 = Unit: 0 Slot: 0 Port: 4 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.16.31.185 = 172.16.31.185
IF-MIB::ifDescr.5 = Unit: 0 Slot: 0 Port: 5 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.24.15.253 = 172.24.15.253
IF-MIB::ifDescr.6 = Unit: 0 Slot: 0 Port: 6 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.24.79.253 = 172.24.79.253
IF-MIB::ifDescr.7 = Unit: 0 Slot: 0 Port: 7 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.24.79.253 = No more variables left in this MIB View
IF-MIB::ifDescr.8 = Unit: 0 Slot: 0 Port: 8 Gigabit - Level 0x6070001
IP-MIB::ipAdEntAddr.172.24.79.253 = No more variables left in this MIB View
IF-MIB::ifDescr.9 = Virtual Interface
IP-MIB::ipAdEntAddr.172.24.79.253 = No more variables left in this MIB View
IF-MIB::ifDescr.9 = No more variables left in this MIB View
Besides getting EOM for the IP-MIB OID, because it has less elements than the IF-MIB OID, I still see the EOM response or not for the IF-MIB OID depending on my fragmentation / maxRepitition param.
I can live with EOMs for the 'shorter' OID (because I wouldn't ever do parallel walks in different tables), but I find it strange to get different results for the same query when using different fragmentations, because the fragmentation is a lower-level protocol issue which I would expect to be hidden by the hlapi.
Add: may be I'm a little bit nitpicking: it's definitely just for the sake of a consistent API. If you know this situation may happen, you can easely get around it.
from pysnmp.proto.rfc1905 import endOfMibView
...
for varBind in varBinds:
name, val = varBind
if val == endOfMibView:
print("Ignore this element!")
else:
print(name.prettyPrint() + ":" + val.prettyPrint(val))
The core SNMP API shouldn't attempt to perform any OID filtering, or you can never satisfy all kinds of filtering needs. I don't think any further work needed on this item.
