qBittorrent icon indicating copy to clipboard operation
qBittorrent copied to clipboard

Memory leak in v5.0.0

Open kieraneglin opened this issue 1 year ago • 50 comments

qBittorrent & operating system versions

qBittorrent: v5.0.0 x64 OS: It's running in an Arch Linux-based Docker container. The host is Unraid 6.12.13 Qt: 6.7.3 Libtorrent: 2.0.10.0 Boost: 1.86.0 OpenSSL: 3.3.2 zlib: 1.3.1

What is the problem?

Based on memory usage, there appears to be a slow leak introduced on my system by the 5.0.0 upgrade. I was running 4.6.7 beforehand (I think) without any issue.

The memory usage constantly increases and appears to be independent from torrent activity overall. There are spikes in memory when a torrent is actively downloading, but the overall trend slowly marches upwards as shown in the usage screenshot below. Restarting instantly fixes the issue in the immediate, but the usage will then start climbing from that point.

I understand that memory leaks SUCK to debug so I'm happy to run any diagnostic/troubleshooting steps you'd like!

Steps to reproduce

  1. Start qBittorrent
  2. Wait and see memory increasing

Additional context

Screenshot 2024-10-04 at 8 23 56 AM

Log(s) & preferences file(s)

watched_folders.json is empty (contents is {}) qBittorrent.conf.txt

kieraneglin avatar Oct 04 '24 16:10 kieraneglin

There is no memory leak. Libtorrent 2.x builds always worked this way. See arvidn/libtorrent#6667 for details. Your best course of action is to set RAM limit for the container.

HanabishiRecca avatar Oct 04 '24 17:10 HanabishiRecca

Sure, but what would the explanation be for this only emerging with the upgrade from 4.6.7 to 5.0.0? Is it purely a reporting issue?

My logging history doesn't go back far enough for me to prove it, but I check my metrics dashboard a few times per week and this issue can be directly correlated with my upgrade to v5. That's not to say that you're wrong! But from where I'm standing this is a novel behaviour.

kieraneglin avatar Oct 04 '24 17:10 kieraneglin

I had the same experience, its using 22.5GB RAM very quickly before I ended the task from task manager (it wouldn't even exit normally and won't open a window to the app). I used 4.6.6-4.6.7 extensively and didn't have this issue before.

x7amod avatar Oct 05 '24 00:10 x7amod

This problem has been known for years now. Does v5 force the use of Libtorrent v2, or can I upgrade to v5 safely using Libtorrent v1.2? I need to switch bittorrent clients every 5 to 10 years because projects somehow seem to screw up. This trend started with uTorrent.

SanderBouwhuis avatar Oct 05 '24 04:10 SanderBouwhuis

@SanderBouwhuis there are 2 variation of each release, for example: 5.0.0 5.0.0 (qt6 lt20)

First one uses libtorrent 1.2 (both uses qt6)

In the next major release of qB (5.1 not 5.0.x), will have a workaround for the lt 2.0 issue.

ArcticGems avatar Oct 05 '24 06:10 ArcticGems

Can confirm. I was on 4.6.7 up until 10/01 and you can very clearly see where the upgrade took place. My Zabbix worker ended up crashing due to the memory usage spike and the memory usage has been erratic ever since.

image

Dekumori avatar Oct 05 '24 08:10 Dekumori

Can confirm same 5.0 docker container issue, RES size increased to 2GB of ram from initial usage of ~20MB for a single inactive torrent running over the period of 5-6 days. Going to stay on 4.6.x branch until this is properly fixed.

TGKx avatar Oct 05 '24 16:10 TGKx

@Dekumori @TGKx What Libtorrent version is mentioned in your qB 5.0 installation: Help->About->Software Used

If Libtorrent 2.0.X, try qB 5.0 release variation with 1.2.X

ArcticGems avatar Oct 05 '24 18:10 ArcticGems

Qt: | 6.6.3 Libtorrent: | 1.2.19.0 Boost: | 1.84.0 OpenSSL: | 3.3.2 zlib: | 1.3.1

This is from docker image: https://github.com/qbittorrent/docker-qbittorrent-nox/pkgs/container/docker-qbittorrent-nox/281473771?tag=5.0.0-1

TGKx avatar Oct 05 '24 19:10 TGKx

@ArcticGems I am on Lib 2.0.X. I'll switch releases. I did mitigate the issue by limiting memory usage in my docker compose yml file. My system is stable, though the container is still unstable. Below is the memory usage since the change. I'll update after a few hours running the alternate release.

image

Dekumori avatar Oct 05 '24 19:10 Dekumori

The problem is definitely there. 4.6.7 was working without an issue. 5.0.0 gets OOM killed on my cluster every 8-9 hours, like a clockwork since the update image

maxim-mityutko avatar Oct 05 '24 21:10 maxim-mityutko

The problem is definitely there. 4.6.7 was working without an issue. 5.0.0 gets OOM killed on my cluster every 8-9 hours, like a clockwork since the update

@maxim-mityutko What Libtorrent version is mentioned in your qB 5.0 installation: Help->About->Software Used

If Libtorrent 2.0.X, try qB 5.0 release variation with 1.2.X

ArcticGems avatar Oct 06 '24 06:10 ArcticGems

Confirmed still occurring after switching to Libtorrent 1.2.19.0 release. It's not maxing out because I put a limit on the docker container.

image

Dekumori avatar Oct 06 '24 09:10 Dekumori

Confirmed still occurring after switching to Libtorrent 1.2.19.0 release.

Finally a useful comment. There is another known issue:

  • #20675
  • #20873

Does it resembles your use pattern?

HanabishiRecca avatar Oct 06 '24 10:10 HanabishiRecca

It looks like both come down to issues with the webui being open. I closed all webui tabs and will monitor for a few hours. If the issue persists, I will restart the container and monitor again. If the issue still persists, I'll restart the server and monitor again. I'll return back with results once the issue is fixed or all of my above steps have been exhausted. I'm happy to try any alternative steps that are suggested as well.

Dekumori avatar Oct 06 '24 23:10 Dekumori

@ArcticGems


qBittorrent was built with the following libraries:

Qt:	6.7.3
Libtorrent:	1.2.19.0
Boost:	1.86.0
OpenSSL:	3.3.2
zlib:	1.3.1.zlib-ng

maxim-mityutko avatar Oct 07 '24 08:10 maxim-mityutko

It looks like both come down to issues with the webui being open. I closed all webui tabs and will monitor for a few hours. If the issue persists, I will restart the container and monitor again. If the issue still persists, I'll restart the server and monitor again. I'll return back with results once the issue is fixed or all of my above steps have been exhausted. I'm happy to try any alternative steps that are suggested as well.

  • The machine from which I usually manage the torrents is turned off for the most part of the day.
  • My instance is running in the pod in Kubernetes, pretty much isolated from anything else
  • I'm using VueTorrent WebUI instead of the native one

maxim-mityutko avatar Oct 07 '24 09:10 maxim-mityutko

Same behaviour in Windows 10 too. Problem started after I'm updating from version installed from file “qbittorrent_4.6.7_lt20_qt6_x64_setup.exe” to the version “qbittorrent_5.0.0_lt20_qt6_x64_setup.exe” image

IRainman avatar Oct 07 '24 15:10 IRainman

Please post full technical details about your system and qBittorent build, as requested by the bug report template. "Me too" comments without details are not helpful.

HanabishiRecca avatar Oct 07 '24 15:10 HanabishiRecca

Thank you.

Problem started after I'm updating from version installed from file “qbittorrent_4.6.7_lt20_qt6_x64_setup.exe” to the version “qbittorrent_5.0.0_lt20_qt6_x64_setup.exe”

Try the regular one instead and see if the problem persists. https://www.fosshub.com/qBittorrent.html?dwl=qbittorrent_5.0.0_x64_setup.exe

HanabishiRecca avatar Oct 07 '24 15:10 HanabishiRecca

Thank you.

Problem started after I'm updating from version installed from file “qbittorrent_4.6.7_lt20_qt6_x64_setup.exe” to the version “qbittorrent_5.0.0_lt20_qt6_x64_setup.exe”

Try the regular one instead and see if the problem persists. https://www.fosshub.com/qBittorrent.html?dwl=qbittorrent_5.0.0_x64_setup.exe

Thank you, but I don't have any reason to use it. I especially use the version with qt6, lib torrent 2.0, enabled memory mapped files, and SQLite for storing data. Currently, I am downgrading to version 4.6.7 and absolutely all goings fine. image

IRainman avatar Oct 07 '24 16:10 IRainman

Reverted to 4.6.7 with Libtorrent 2 and the issue does NOT persist. Usage in the graph correlates perfectly with new torrents being downloaded.

Qt: 6.7.2 Libtorrent: 2.0.10.0 Boost: 1.86.0 OpenSSL: 3.3.2 zlib: 1.3.1

Screenshot 2024-10-07 at 9 33 22 AM

kieraneglin avatar Oct 07 '24 16:10 kieraneglin

I use the version 1.2 libtorrent of 5.0 qbt, but it cost about 20gb physics memory and all the virtual memory about 30 gb,50 gb in total. And I installed 4.6.6 back, and it works much better than 5.0.0ver.

starobots avatar Oct 07 '24 17:10 starobots

Another possible test: try to reset the client to default settings. I.e. delete existing client config (make a backup if you want). The config location per platform is described in the wiki.

Also we need:

  1. Reproduction steps. Exactly what you do with the client, maybe some patterns you've noticed.
  2. Logs. Any info as much as possible.
  3. Crash reports, dumps and backtraces, if any.

P.S. If everyone going to flip the table and roll back to older versions, the problem would never be fixed.

HanabishiRecca avatar Oct 07 '24 18:10 HanabishiRecca

P.S. If everyone going to flip the table and roll back to older versions, the problem would never be fixed.

I'm not sure if this is your intent, but many of your replies are coming off as antagonistic. I've rolled back as a troubleshooting step to gather more details and not because I'm throwing in the towel. As said in my original post I'm more than happy to put in some work to help identify the root cause.

In any case, I'll spin up a new v5.0.0 Docker container which will provide me with a completely blank slate and let you know what I see!

kieraneglin avatar Oct 07 '24 19:10 kieraneglin

I've rolled back as a troubleshooting step to gather more details and not because I'm throwing in the towel. As said in my original post I'm more than happy to put in some work to help identify the root cause.

Glad to hear. Due to the comment section being kinda spammy, it's easier to lose track. That's because 3 comments in a row above my message looked like a throw:

I am downgrading to version 4.6.7 Reverted to 4.6.7 I installed 4.6.6 back


I'll spin up a new v5.0.0 Docker container which will provide me with a completely blank slate and let you know what I see!

Sure. It's a crucial step now to pinpoint the issue and make it reproducible.


And yes, my initial assumption (https://github.com/qbittorrent/qBittorrent/issues/21502#issuecomment-2394182076) seems to be wrong now. We just so used to such reports at this point, that anything about "memory leak" triggers that association by reflex.

HanabishiRecca avatar Oct 07 '24 19:10 HanabishiRecca

Update: A completely fresh qbit 5.0.0 install with no torrents has been slowly increasing in memory usage. It's important to note the scale here - this is showing 62 MB to ~85MB over the course of 2 hours. Far, far slower than the prior ~500 MB per hour I was seeing before. These numbers may be too small as to be useful but I figured I'd update.

I'll keep it running and see what I find

Screenshot 2024-10-07 at 3 27 46 PM

kieraneglin avatar Oct 07 '24 22:10 kieraneglin

@stalkerok I know you maintain big amount of torrents. Have you already switched to qBittorrent v5.0?

glassez avatar Oct 08 '24 04:10 glassez

hello all, i'm here to confirm this bug, i'm fully aware of the mmap issue w/ LT20, actually i'm one of the people who pleaded to have the LT1.2 option with QBT 5.0 so I'm still using QBT 5.0 with LT1.2 ...

if anyone can recommend a tool to memory profile the client while it's running so I can provide more details? one that works on docker containers/qbitorrent nox.. it's so severe that the machine actually uses up the entire swap memory, this happened on two separate linux machines, one hasn't crashed yet thankfully, but will downgrade to 4.6.7 with lt1.2 for the meantime...

Machine 1: image

Machine 2: image

vincejv avatar Oct 08 '24 15:10 vincejv

Also im guessing machine 1 crashed first due to it having a higher number of torrents active..

vincejv avatar Oct 08 '24 15:10 vincejv