qBittorrent
qBittorrent copied to clipboard
qbittorrent
v4.4.X.X x64 - Insane memory usage(Linux)
qBittorrent & operating system versions
qBittorrent: v4.4.3.1 x64 OS: Debian 11 Qt: 6.3.0 Libtorrent: 2.0.6.0 32 GB RAM Additional: qBit is running in a docker container using the latest image from linuxserver.io that uses alpine linux.
What is the problem?
I should preface this by saying that I've had this problem for a while, I just kinda hoped it would fix itself, but i got tired of waiting.
qBit will gladly eat up 16+ GB if memory until it gets killed by OOM killer (Below is a pic of it using 10 GB of memory:
I currently have 1k torrents active, this problem seem to have appeared after upgrading to version 4.4.X.X, it seems similar to #17309.
Additionally torrents I've added or removed will sometimes disappear after qBit is restarted by docker or OOM killer which makes qBit pretty much unusable.
I've tried using a few other qBit images from docker hub but they all result in the same issue.
Steps to reproduce
- Add 1K torrents to qBit.
- Wait a couple of hours.
- qBit will eat all your memory.
- OOM killer will kill qBit.
- The WebUI will show the error "qBittorrent client is not reachable"
Additional context
I've put a limit of 6 GB memory usage on the docker container itself, these are the docker container logs every time it gets restarted:
2022-08-31T00:23:55.946709627Z
2022-08-31T00:23:55.947000814Z ******** Information ********
2022-08-31T00:23:55.947010515Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T00:26:01.496767580Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T02:18:07.172354902Z
2022-08-31T02:18:07.172894002Z ******** Information ********
2022-08-31T02:18:07.173009875Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T02:18:56.446415076Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T04:20:12.130889920Z
2022-08-31T04:20:12.131330667Z ******** Information ********
2022-08-31T04:20:12.131359503Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T04:20:47.784849538Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T06:20:27.451137030Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T06:20:27.453306236Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T06:34:14.645288197Z
2022-08-31T06:34:14.645656521Z ******** Information ********
2022-08-31T06:34:14.645669663Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T06:35:05.353536265Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T06:35:05.380552648Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T06:35:05.380606846Z qt.network.http2: stream 3 finished with error: "Connection closed"
2022-08-31T08:28:18.837927938Z
2022-08-31T08:28:18.838172681Z ******** Information ********
2022-08-31T08:28:18.838187366Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T08:29:14.221791584Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T10:28:24.067455827Z
2022-08-31T10:28:24.067683581Z ******** Information ********
2022-08-31T10:28:24.067692607Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T10:29:26.379433784Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T12:26:29.100403377Z
2022-08-31T12:26:29.101100365Z ******** Information ********
2022-08-31T12:26:29.101136964Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T12:27:28.103326204Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T14:28:34.599028389Z
2022-08-31T14:28:34.599643332Z ******** Information ********
2022-08-31T14:28:34.599688616Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T14:29:31.523323642Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T16:32:36.193621877Z
2022-08-31T16:32:36.194006588Z ******** Information ********
2022-08-31T16:32:36.194023346Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T16:33:36.700893552Z qt.network.http2: stream 1 finished with error: "Connection closed"
2022-08-31T18:30:39.131304315Z
2022-08-31T18:30:39.132063194Z ******** Information ********
2022-08-31T18:30:39.132095867Z To control qBittorrent, access the WebUI at: http://localhost:8080
2022-08-31T18:31:27.655796523Z qt.network.http2: stream 1 finished with error: "Connection closed"
Log(s) & preferences file(s)
My config file: qBittorrent_conf.txt
The logs file just shows a bunch of torrents being restored and my RSS feeds being downloaded. (Didn't add it since I'd have to redact all of it anyways)
@NUReq45DTz7c
qBittorrent: v4.4.3.1 x64 Libtorrent: 2.0.6.0
Newest is qBittorrent v4.4.5 and libtorrent v1.2.17+ or v2.0.7+, so please try using newest qBittorrent and libtorrent before reporting issues.
Official qBittorrent News page even mentions to try to use libtorrent v1.2 if anyone is experiencing issues with libtorrent v2.
@NUReq45DTz7c
qBittorrent: v4.4.3.1 x64 Libtorrent: 2.0.6.0
Newest is qBittorrent v4.4.5 and libtorrent v1.2.17+ or v2.0.7+, so please try using newest qBittorrent and libtorrent before reporting issues.
Official qBittorrent News page even mentions to try to use libtorrent v1.2 if anyone is experiencing issues with libtorrent v2.
Hi, Since I'm using a pre-made image from docker hub I don't control the versions they use, but I'll defo hit them up and ask them to update their image. Thanks.
Okay the image has been updated to 4.4.5 and libtorrent 2.0.7.0 ill report back if that changes anything. I swear if I just had to wait a little bit more for the issue to fix itself after I've been waiting for so long...
Kinda known problem with libtorrent 2.0.x which is being worked on on the libtorrent side. That's why the official builds switched to libtorrent 1.2.x by default while providing libtorrent 2.0.x as the alternative. Try our AppImage as a test.
Same problem & can't find "max limit for memory usage" parameter.
qBittorrent: v4.4.5 x64 OS: Windows 10 x64 21H2 Qt: 6.3.0 Libtorrent: 1.2.17.0 32 GB RAM
Unfortunately qBit still seems to use a bunch of memory until OOM killer kills it :( qBittorrent: v4.4.5 x64 OS: Debian 11 Qt: 6.3.1 Libtorrent: 2.0.7.0 32 GB RAM
As before the log file doesn't show anything useful, my settings haven't changed.
@NUReq45DTz7c Try to reproduce with libtorrent v1.2.17 instead of v2.0.7.
The machine qBit is running on is a headless machine, is there a way to enable the webui via cli when using the appimage (i was thinking of using the appimage as it seems to run v1.2.17 and I don't control the versions the docker image uses)? i can only see there's an argument for changing the webui port but that doesn't seem to enable the webui itself.
is there a way to enable the webui via cli when using the appimage
Unfortunately the appimage is using gui libraries so it probably can't work under headless.
However, can you try setting vfs_cache_pressure to 200 and seeing what happens (with current docker images)? When memory starts filling up due to file caching the kernel should start dropping the file cache from memory instead of trying to swap out programs. So it should prevent the OOM killer being triggered. The RAM usage may still grow though.
You can do it with:
sudo sysctl -w vm.vfs_cache_pressure=200
NOTE: The above change is temporary. It will be lost with a reboot.
Hi, Thanks for the help, I've set vfs_cache_pressure to 200, I'll report back with what happens.
OS: Windows 7 x64 Installer: qbittorrent_4.4.5_x64_setup.exe
I have updated from 4.4.3.1 to 4.4.5 and got a memory over consumption ended with crash. Got this 2 times over ~12 hours with 4GB Limit. The previous version does not crash for weeks.
@andry81
What libtorrent or installer did you use before? What settings do you use to set 4GB limit?
What libtorrent or installer did you use before?
qbittorrent_4.4.3.1_x64_setup.exe
What settings do you use to set 4GB limit?
VirtualBox 6.1.36, 2 cores, enabled I/O APIC+PAE/NX+Nested VT-x/AMD-V
@andry81
Then you where using libtorrent v2.0+ before and starting with qBittorrent v4.4.5 you where using libtorrent v1.2.17+. They work differently and have different Advanced settings in newest qBittorrent.
You can see version info of your used dependencies if open qBittorrent - Help - About - Software Used.
It's even written on the official News Page that you may continue using libtorrent v2.0.x if you previously didn't experience issues:
https://www.qbittorrent.org/news.php NOTE: The default builds for all OSs switched to libtorrent 1.2.x (RC_1_2) from 2.0.x (RC_2_0). Builds for libtorrent 2.0.x are also offered and are tagged with RC_2_0. The switch happened due to user demand and perceived performance issues. If until now you didn't experience any performance issue then go ahead and use the RC_2_0 builds.
Original opening post of this issue was by user who was using libtorrent v2.0.6+ and you're using libtorrent v1.2.17+, totally different OS and different libtorrent versions cache and manage files differently aswell. So this is bit offtopic if different users use different Operating System, qBittorrent and libtorrent and experience memory usage totally differently.
@andry81 @PriitUring I hid your comments as "offtopic" because the original user here is using Linux. IIRC there are other issues discussing windows ram usage. Let's not drown this issue with Windows problems/solutions.
Unfortunately that didn't change anything either :(
Did it still kill qbittorrent?
Maybe try vfs_cache_pressure with a more extreme setting like 500?
@andry81 @PriitUring I hid your comments as "offtopic" because the original user here is using Linux. IIRC there are other issues discussing windows ram usage. Let's not drown this issue with Windows problems/solutions.
Then this older comment by @FiTADiNE should be hidden aswell: https://github.com/qbittorrent/qBittorrent/issues/17650#issuecomment-1236075089
Maybe good to mention Linux and libtorrent v2.0.x on the issue title? Otherwise more users might come here to comment about their Windows and libtorrent v1.2 related issues.
Did it still kill qbittorrent? Maybe try vfs_cache_pressure with a more extreme setting like
500?
Yeah it did, I've disabled the ram limit on the docker container and I'll try with it set to 500.
Unfortunately that didn't change anything either, qBit is now happily using 21GB of memory :(
Any other things I can try? Currently I'm using Transmission but I'm missing a bunch of features, I really wanna get back to using qBittorrent.
Currently I'm using Transmission but I'm missing a bunch of features, I really wanna get back to using qBittorrent.
Why don't try to return back to 4.4.3.1? It does work for Windows and might work for Linux.
Some other suggestions:
Diagnostics
When RAM usage is high issue the command free -m and post the results here. It should tell us how much RAM is used for buffers/cache.
Possible solutions
- In qbittorrent: Options->Advanced Settings->Enable OS cache (uncheck it)
- Put qbittorrent in a cgroup with memory limit. Instructions
Why don't try to return back to 4.4.3.1? It does work for Windows and might work for Linux.
I tried that after the release of 4.4.X.X but I believe I had to recheck all my torrents after downgrading so I ended up using the latest version instead as rechecking all my torrents would take forever.
When RAM usage is high issue the command free -m and post the results here. It should tell us how much RAM is used for buffers/cache.
I'll try to catch it when it's using a bunch of memory and post the result.
In qbittorrent: Options->Advanced Settings->Enable OS cache (uncheck it)
I've done that now, we'll see if that helps anything.
Put qbittorrent in a cgroup with memory limit. Instructions
I can try, is there really a difference between putting qbit in a cgroup versus limiting the memory usage via docker? What happens to the qbit process when it reaches its memory limit while running in a cgroup?
I can try, is there really a difference between putting qbit in a cgroup versus limiting the memory usage via docker?
I am not well versed with docker. It is possible that docker's memory limiting uses cgroups behind the scenes.
What happens to the qbit process when it reaches its memory limit while running in a cgroup?
If the extra memory is used for OS cache/buffers then it should evict the cache more aggressively to reclaim memory. If not, then (I suppose) the OS might kill processes in the same cgroup to stay below the limit (aka it will kill qbittorrent).
When RAM usage is high issue the command
free -mand post the results here
Here's the output of free -m when qBit was using 10 GB of RAM:
total used free shared buff/cache available
Mem: 31756 29876 1122 178 758 1191
Swap: 4095 3948 147
Note that I disabled OS cache in qBit when running this test. I'll try using cgroups next and see what happens.
Apparently the caches are only 758MB in size. So the 10GB is used by qbt directly. Also it seems that 29GB of RAM are in use. Does the server run other applications apart from qbt?
Note that I disabled OS cache in qBit when running this test.
My mistake for not clarifying. I would like to see free -m when OS cache is enabled.
Notice the value in column buff/cache. If it is close to qbt's RAM, try "dropping the caches".
Run free && sync && echo 3 > /proc/sys/vm/drop_caches && free
It does:
- Prints the memory usage before dropping caches
- Writes cached data to disk
- drops caches
- Prints the memory usage after dropping caches
What are your max # of open files set to witih QBT? mines below 60 and my resident memory is ~5800MB.
Settings -> Advanced -> File Pool Size
Virtual memory is way different than resident. resident is what you care about.
I tried that after the release of 4.4.X.X but I believe I had to recheck all my torrents after downgrading so I ended up using the latest version instead as rechecking all my torrents would take forever.
I've downgraded and it works as is. Yes, not so fast but in the same time is not so long. Anyway, you can backup all your configs and fastresume files manually before downgrade at any time.
Also it seems that 29GB of RAM are in use. Does the server run other applications apart from qbt?
Yup, it runs other stuff like Sonarr, Radarr and Transmission, though most of it is probably used as cache for ZFS.
My mistake for not clarifying. I would like to see free -m when OS cache is enabled.
Okay, will enable OS cache and do it again.
Notice the value in column buff/cache. If it is close to qbt's RAM, try "dropping the caches".
I'll try this too.
I've downgraded and it works as is. Yes, not so fast but in the same time is not so long. Anyway, you can backup all your configs and fastresume files manually before downgrade at any time.
I'll probably downgrade as a last resort.
@sledgehammer999 here is the output of free -m when qBit is using 7 GB of RAM with OS cache enabled:
total used free shared buff/cache available
Mem: 31756 28796 2426 105 533 2507
Swap: 4095 4066 28
Here's the output of the "drop cache" command:
total used free shared buff/cache available
Mem: 32518416 29873912 2166968 107944 477536 2122036
Swap: 4194168 4194096 72
total used free shared buff/cache available
Mem: 32518416 14975340 17108644 108200 434432 17042092
Swap: 4194168 4194092 76
On a side note it seems it mostly "released" the cache ZFS had.
@thisisnotmyrealname it's set to 5000, I believe that's the default.
As suggested by @thisisnotmyrealname, I drastically lowered the number of open files and it seems to have fixed it for me 👍🏻
As suggested by @thisisnotmyrealname, I drastically lowered the number of open files and it seems to have fixed it for me 👍🏻
Hmm okay thanks, I'll try it out.
As suggested by @thisisnotmyrealname, I drastically lowered the number of open files and it seems to have fixed it for me 👍🏻
Tried changing it to 50, qBit is atm consuming 15 GB of ram :c Anything else I can try?
Try the qbittorent docker image provided by hotio. Not sure if linking is allowed, but do a search and you'll find it. The release version with 4.4.5 uses libtorrent 1.2 and so far, I'm not seeing insane memory usage. I'm using Debian 11 as my docker host.
