scapy icon indicating copy to clipboard operation
scapy copied to clipboard

[enhancement] Async Pcap(Ng)Reader implementations

Open calligraf0 opened this issue 1 year ago • 0 comments

Hi, first time opening an issue hope I do everything right :smiley: , I choose not to go for the issue template as it seemed out of context for this since it's not a bug but a proposal.

Intro

As for the current state of Scapy there is no async-aware implementation for the PcapReader/PcapNgReader classes which may instead be useful when parsing many large .pcap* files using Scapy.

Solution?

I am not too familiar with the codebase but I tried to put together a horribly hacked version of possible implementation: at https://github.com/calligraf0/scapy/tree/async-pcap-reader (there is also a junkyard.py file which contains a "benchmark" which currently still lags behind the multithread sync implementation)

The main limitations I stumbled across right now are:

  • this obviously requires a python version which supports asyncio
  • lack of gzip support (the code I smashed together currently does not handle gzip compressed files, as I couldn't find a valid "drop in replacement" for gzip which supports async)
  • pcap/cap only support (I only implemented the bare minimum, so no PcapNgReader for now)
  • ugly defer of header parsing on first packet read (because of async, I couldn't use aiofiles in the __init__ method, a possible solution could be using the AsyncMixin class / rework the way the file is handled?)
  • I am not a great programmer :sweat_smile: (that has no immediate solution :stuck_out_tongue_closed_eyes: )

Questions

  • Is this a valid/interesting enhancement proposal? If so, is the solution I PoC'ed together a viable way (providing an extra class AsyncPcapReader instead of modifying the RawPcapReader class to also support async)?
  • I am happy to try give this a better shot, are there any pointers on what to improve/rework?

calligraf0 avatar Aug 16 '24 14:08 calligraf0