arvidn
Multithread network io
Please provide the following information
libtorrent version (or branch): 1.2 from master
platform/architecture: arch linux 6.2, qBittorrent 4.6.0 aplha
I have ~120k torrents and 3x1Gb connections. When torrents starting their announces, i see about 60k connections (there are only 10k simultaneous announces in qBittorrent settings) and upload speed drops a lot. One of qBittorrent threads eats 100% cpu most of the time. I don’t know for sure, but it seems that all network traffic is processed in one thread. It would be great to have several threads for this, such as for disk io.
To assert my suspicion, I divided the torrents into 2 clients and now all channels are full used most of the time. One thread from each client still eats 100% cpu, but this no longer has a significant effect.
Before (1 qBittorrent client):
After (2 qBittorrent clients):
Connections statistic:
In libtorrent the way to have multiple threads processing network traffic is to instantiate multiple session objects and add torrents to them. Each session object maintains one network thread to avoid excess synchronization as happens when juggling multiple threads.
The client you are using doesn't seem to support starting up multiple sessions.
supporting multiple network threads is something I've looked into on and off for a long time. There are quite a lot of dependencies between peers and torrents that would be quite complicated to tease apart. Maybe if I could drop certain features (like bandwidth limiting), it might make it a bit simpler.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
Just another data point, this has also become the bottleneck for me. I'm trying to seed less than 5 torrents on a 10 Gbit/s network, and the network thread becomes a bottleneck, using 100% of a CPU at ~6 Gbit/s. Reducing the number of leechers to less than 200 can take it to ~7 Gbit/s, but still can't saturate the network.
My CPU is an i7-1355U with pretty good single-thread performance. I could get a more powerful CPU to get up to 30% increased single-thread performance, but not without at least increasing my TDP by 4x.
libtorrent version: 2.0.11 qBittorrent version: 4.6.7
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.