[enhancement] Async Pcap(Ng)Reader implementations
Hi, first time opening an issue hope I do everything right :smiley: , I choose not to go for the issue template as it seemed out of context for this since it's not a bug but a proposal.
Intro
As for the current state of Scapy there is no async-aware implementation for the PcapReader/PcapNgReader classes which may instead be useful when parsing many large .pcap* files using Scapy.
Solution?
I am not too familiar with the codebase but I tried to put together a horribly hacked version of possible implementation: at https://github.com/calligraf0/scapy/tree/async-pcap-reader (there is also a junkyard.py file which contains a "benchmark" which currently still lags behind the multithread sync implementation)
The main limitations I stumbled across right now are:
- this obviously requires a python version which supports asyncio
- lack of gzip support (the code I smashed together currently does not handle gzip compressed files, as I couldn't find a valid "drop in replacement" for gzip which supports async)
- pcap/cap only support (I only implemented the bare minimum, so no PcapNgReader for now)
- ugly defer of header parsing on first packet read (because of async, I couldn't use aiofiles in the
__init__method, a possible solution could be using the AsyncMixin class / rework the way the file is handled?) - I am not a great programmer :sweat_smile: (that has no immediate solution :stuck_out_tongue_closed_eyes: )
Questions
- Is this a valid/interesting enhancement proposal? If so, is the solution I PoC'ed together a viable way (providing an extra class AsyncPcapReader instead of modifying the RawPcapReader class to also support async)?
- I am happy to try give this a better shot, are there any pointers on what to improve/rework?