torrentool icon indicating copy to clipboard operation
torrentool copied to clipboard

Improve bencode speed

Open ngosang opened this issue 4 years ago • 3 comments

https://pypi.org/project/modern-bencode/

decode torrentool_bencode 29.49319815635681 s decode modern-bencode 13.25705099105835 s

Booth libraries are using pure Python code. Maybe you can improve your performance.

def read_torrentool_bencode(torrent_file):
    metadata = torrentool.bencode.Bencode().decode(torrent_file)
    return metadata

def read_modern_bencode(torrent_file):
    metadata = bencode.decode(torrent_file)
    return metadata

torrent_filepath = "aaa.torrent"
with open(torrent_filepath, 'rb') as inp:
    torrent_file = inp.read()

    start = time.time()
    for i in range(0, 100000):
        metadata = read_torrentool_bencode(torrent_file)
    end = time.time()
    print("read_torrentool_bencode", end - start)

    start = time.time()
    for i in range(0, 100000):
        metadata = read_modern_bencode(torrent_file)
    end = time.time()
    print("read_modern_bencode", end - start)

ngosang avatar Oct 02 '21 18:10 ngosang

Thank you.

Maybe you can improve your performance.

It might be so, yet do you have any particular reason to strive for numbers?

idlesign avatar Oct 04 '21 01:10 idlesign

I'm working in my own torrent indexer site, so, I tested several Python libraries. I'm indexing thousands of torrents and serving hundred thousand torrent scrape requests. In this use case performance is important but It's not a common case.

  • decode torrentool_bencode 29.49319815635681 s
  • decode modern-bencode 13.25705099105835 s
  • decode better-bencode 0.98 s

Finally I'm using better-bencode for my project but this is a nice project with more functionality. Maybe I use it for other tasks. This issue is not a priority for me, but can be useful for others. https://github.com/kosqx/better-bencode

ngosang avatar Oct 04 '21 12:10 ngosang

I'm indexing thousands of torrents and serving hundred thousand torrent scrape requests.

I see. Thank you.

decode better-bencode 0.98 s

That's С, not Python.

idlesign avatar Oct 05 '21 03:10 idlesign

Considered closed. Feel free to reopen if required.

idlesign avatar Jan 20 '23 05:01 idlesign