scapy
scapy copied to clipboard
secdev
Topic: Scapy slow startup because of too many routes
Brief description
Hi
I have multiple (2000) IPv6 addresses on vlan interface. This causes scapy to starts few minutes.
Problem is that for each address, list of addressess on its interface is gathered with function:
cset = scapy.utils6.construct_source_candidate_set(d, dp, devaddrs) (in read_routes6)
And it works slowly as inside there is sorting mechanism:
cset.sort(key=cmp_to_key(cset_sort)) in construct_source_candidate_set
Having multiple addresses on interface potentially it would be enough to run construct_source_candidate_set once for global address, as it always gives the same result.
On the other hand maybe sorting mechanism is not needed? Without it scapy starts immediately.
Scapy version
2.4.0rc5
Python version
2.7.12
Operating system
Linux 4.15.0-43-generic #46~16.04.1-Ubuntu SMP
Additional environment information
No response
How to reproduce
Add >2000 addresses under interface. Run scapy.
Actual result
ip addr show dev eth1.47 | grep inet6 | wc -l
2000
time scapy
INFO: Can't import PyX. Won't be able to use psdump() or pdfdump().
WARNING: IPython not available. Using standard Python shell instead.
AutoCompletion, History are disabled.
aSPY//YASa
apyyyyCY//////////YCa |
sY//////YSpcs scpCY//Pp | Welcome to Scapy
ayp ayyyyyyySCP//Pp syY//C | Version 2.4.0rc5
AYAsAYYYYYYYY///Ps cY//S |
pCCCCY//p cSSps y//Y | https://github.com/secdev/scapy
SPPPP///a pP///AC//Y |
A//A cyP////C | Have fun!
p///Ac sC///a |
P////YCpc A//A | Craft me if you can.
scccccp///pSP///p p//Y | -- IPv6 layer
sY/////////y caa S//P |
cayCyayP//Ya pY/Ya
sY/PsY////YCc aC//Yp
sc sccaCY//PCypaapyCP//YSs
spCPY//////YPSps
ccaacs
>>>
real 4m6.562s
user 2m8.642s
sys 0m1.886s
Expected result
ip addr show dev eth1.47 | grep inet6 | wc -l
2000
sed -i "s/ cset.sort/ #cset.sort/g" /usr/local/lib/python2.7/dist-packages/scapy/utils6.py
time scapy
INFO: Can't import PyX. Won't be able to use psdump() or pdfdump().
WARNING: IPython not available. Using standard Python shell instead.
AutoCompletion, History are disabled.
aSPY//YASa
apyyyyCY//////////YCa |
sY//////YSpcs scpCY//Pp | Welcome to Scapy
ayp ayyyyyyySCP//Pp syY//C | Version 2.4.0rc5
AYAsAYYYYYYYY///Ps cY//S |
pCCCCY//p cSSps y//Y | https://github.com/secdev/scapy
SPPPP///a pP///AC//Y |
A//A cyP////C | Have fun!
p///Ac sC///a |
P////YCpc A//A | Craft packets like I craft my beer.
scccccp///pSP///p p//Y | -- Jean De Clerck
sY/////////y caa S//P |
cayCyayP//Ya pY/Ya
sY/PsY////YCc aC//Yp
sc sccaCY//PCypaapyCP//YSs
spCPY//////YPSps
ccaacs
>>>
real 0m3.080s
user 0m2.194s
sys 0m0.164s
Related resources
No response
Kind of a duplicate of https://github.com/secdev/scapy/issues/2133 but thanks for reminding us about this issue.
Hi, I tested this case in my environment, and I don't think, that using ip route instead of /proc is a clue. In my case ip route works even slower than /proc/net/ipv6_route with 2000 IPv6 addresses:
$ time ip -6 route list table all | wc -l
2009
real 0m0.097s
user 0m0.028s
sys 0m0.067s
$ time cat /proc/net/ipv6_route | wc -l
2010
real 0m0.031s
user 0m0.001s
sys 0m0.030s
So I guess that problem is in very slow sorting method
cset.sort(key=cmp_to_key(cset_sort))
in construct_source_candidate_set method.
I also verified scapy starting time with 2000 IPv4 addresses, and I didn't have such problem. Looking through the code, I didn't see any sorting method involved in read_routes().
