Nexcloud client memory consumption is too high

[Wenzel]

Hi,

i just noticed that my nextcloud client is using too much memory on my laptop. He is just watching a few files and really shouldn't consume almost 400MB of my RAM.

Details: OS: Fedora 27 nextcloud is installed via package manager: nextcloud-client version: 2.3.3

If anyone has the same issue, please give a feedback here.

Thanks !

[ErikThorsell]

I have the same issue. Nextcloud version: 2.3.3-1, installed from AUR OS: Arch Linux (uname -r == 4.15.15-1-ARCH)

I attributed the issue to me having synced a lot of files recently, but after the syncing was done the high memory usage remained. Heck, Nextcloud is using more than twice the memory of Firefox, that should not need to happen. Right?

[camilasan]

Agreed, we can improve. That should not need to happen.

[Wenzel]

@camilasan can we ping Nextcloud devs to have some idea how to debug this ?

How can we provide you more information and help you find the bug ?

[camilasan]

There are a few things I could think of: clone the repo and debug it on Qt Creator. What were you doing when you first notice the memory usage (syncing, sharing..)? I would also look in the logs based on that info and maybe open the client from the terminal to check for any weird warnings/errors and repeat what you did that might have increased the memory usage. But I won't have time to look at this right now. So yes, please, ping people on irc (#nextcloud-dev / #nextcloud-client), or in the forums.

[affinityv]

This is still a problem 2.5.1.61652 on Windows Server 2012 R2 was using close to 2 GB after being left running for some time.

[bill-mcgonigle]

On Debian Buster, Nextcloud 2.5.1git, I am at 280MB RSS at launch, which rapidly grows to 700MB RSS within a minute or so. After the machine has been up a while I've seen it up at 1.8GB RSS so I usually have to quit it and run Nextcloud occasionally, which I'd rather not do.

I am using pmap -x (pidof nextcloud) where the parens are backticks, but I can't get the markdown escape sequences to work right now.

What is the design target for memory usage?

[Victor-Morel]

On Debian sid, Nextcloud 2.5.2 is using up to 3GB of RAM. I know I have many files, but it is clearly impractical :/

[jpeeters]

Hi,

Thx for developping NextCloud. This is a great tool.

However, I have Nextcloud client running on Arch for 2 days and it consumes more than 12GB of RAM (32GB of RAM available on this machine) and counts dozen of subprocesses/threads.

Is that normal ? Have you already have tried a valgrind run on your code ?

Best, Julien

[janprzy]

The Nextcloud client is currently using 1.55 GB on my Mac, so definitely not exclusive to Linux

[ConstantMown]

Same here, Nextcloud desktop version 2.3.3 on MacOS 10.14.6, uptime about 15-16 days, memory usage is about 1.4 GB, yes the unit is gigabytes. Update to 2.5.3, just after update memory usage was 41.6 MB, unit is megabytes. About four hours after update memory usage is 128.3 MB.

1st edit: In my opinion there is memory leak(s) in desktop client. 2nd edit: After eight hours the memory usage is 218.7 MB. I have not added or removed files. In Finder the size of the managed folder size 45.1 MB and it has 35 "items".

[omni-vi]

Same here as well. MacOS client 2.5.3release (build 20190724). Client started with a memory consumption for just under 50MB - after 4 days it is now at just under 450MB.

[ConstantMown]

I've been running script that runs "heap -s NextCloudClientPID" command once per Nextcloud client process, and then it sleeps for 24 hours.

There has been no new uploads into my Nextcloud, and the number of files and folders is about twenty.

I've attached the two files to this comment Below are some differences between these two files

--- nextcloud.1522.2019-08-27.txt 2019-08-27 09:42:01.000000000 +0300 +++ nextcloud.1522.2019-09-01.txt 2019-09-01 09:42:30.000000000 +0300

-Date/Time: 2019-08-27 09:42:01.355 +0300 +Date/Time: 2019-09-01 09:42:20.719 +0300 Launch Time: 2019-08-27 09:37:49.378 +0300 OS Version: Mac OS X 10.14.6 (18G95) Report Version: 7 Analysis Tool: /Applications/Xcode.app/Contents/Developer/usr/bin/heap Analysis Tool Version: Xcode 10.3 (10G8)

-Physical footprint: 40.8M -Physical footprint (peak): 47.2M +Physical footprint: 2.6G +Physical footprint (peak): 2.6G [snip] -All zones: 186228 nodes (25357696 bytes) +All zones: 14461382 nodes (2732489072 bytes)

nextcloud.1522.2019-08-27.txt nextcloud.1522.2019-09-01.txt

Could @Nextcloud or for example @rullzer take a look at this issue? Q: Why did you pick @rullzer? A: Because he was the last person to update Nextcloud client sources in github.

[bill-mcgonigle]

@ConstantMown : any chance you could bisect the versions? Or run it under Heob or Valgrind? e.g. https://doc.qt.io/qtcreator/creator-heob.html

[ConstantMown]

I'm running MacOS Mohave, version 10.14.6 (18G95). Mohave is even worse than High Sierra, version 10.13, when it comes to "Random Apple operating system developer knows better than the end user, what the end user wants to do" mentality, e.g. Mohave prevents doing bunch of stuff, in an effort to keep end user safe, but I digress.

1. Sorry I don't understand what you mean by "bisect the versions".

2. Downloaded heob sources from github and trying to build it results in

$ make all i686-w64-mingw32-windres -DHEOB_VER_STR=\"3.2-dev-20190902\" -DHEOB_VER_NUM=3,2,0,99 -DHEOB_PRERELEASE=1 -DHEOB_COPYRIGHT_YEARS=\"2014-2019\" heob-ver.rc heob-ver32.o make: i686-w64-mingw32-windres: No such file or directory make: *** [heob-ver32.o] Error 1

3. trying to run nextcloud in valgrind gives this error message:

$ valgrind /Applications/nextcloud.app/Contents/MacOS/nextcloud ==6365== Memcheck, a memory error detector ==6365== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==6365== Using Valgrind-3.16.0.GIT and LibVEX; rerun with -h for copyright info ==6365== Command: /Applications/nextcloud.app/Contents/MacOS/nextcloud ==6365== --6365-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option --6365-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 2 times) --6365-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 4 times) ==6365== Thread 2: ==6365== Invalid read of size 4 ==6365== at 0x10CBE2808: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== by 0x10CBE2497: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== by 0x10CBE23FC: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== Address 0x18 is not stack'd, malloc'd or (recently) free'd ==6365== ==6365== ==6365== Process terminating with default action of signal 11 (SIGSEGV) ==6365== Access not within mapped region at address 0x18 ==6365== at 0x10CBE2808: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== by 0x10CBE2497: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== by 0x10CBE23FC: ??? (in /usr/lib/system/libsystem_pthread.dylib) ==6365== If you believe this happened as a result of a stack ==6365== overflow in your program's main thread (unlikely but ==6365== possible), you can try to increase the size of the ==6365== main thread stack using the --main-stacksize= flag. ==6365== The main thread stack size used in this run was 8388608. --6365:0:schedule VG_(sema_down): read returned -4 ==6365== ==6365== HEAP SUMMARY: ==6365== in use at exit: 0 bytes in 0 blocks ==6365== total heap usage: 0 allocs, 0 frees, 0 bytes allocated ==6365== ==6365== All heap blocks were freed -- no leaks are possible ==6365== ==6365== For lists of detected and suppressed errors, rerun with: -s ==6365== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 4988 from 21) Segmentation fault: 11 $

FWIW running "/Applications/nextcloud.app/Contents/MacOS/nextcloud" starts the Nextcloud Desktop app.

I can create a new virtual machine, running a linux desktop of sorts, probably Ubuntu 18.04.3 LTS and try to run Nextcloud desktop in valgrind or Qt5+heob if valgrind route does not work.

[ConstantMown]

Hello @bill-mcgonigle and @nextcloud I ran

valgrind --leak-check=full --show-leak-kinds=all --trace-children=yes --log-file='nextcloud-desktop.valgrind.%p.log' $(which nextcloud)

in a Ubuntu 18.04.3 LTS desktop. I made a new user into my Nextcloud server and used that in the Nextcloud desktop. The desktop client was installed from Ubuntu repository, which was added 'sudo add-apt-repository ppa:nextcloud-devs/client' command. After about ten minutes, I stopped the nextcloud desktop.

The before mentioned command made about 67 megabytes of logs.

$ du -sh *.log
4.0K	nextcloud-desktop.valgrind.1.log
25M	nextcloud-desktop.valgrind.2588.log
392K	nextcloud-desktop.valgrind.2612.log
8.0K	nextcloud-desktop.valgrind.2613.log
9.2M	nextcloud-desktop.valgrind.2615.log
9.2M	nextcloud-desktop.valgrind.2641.log
25M	nextcloud-desktop.valgrind.2728.log
$ 

The logs have been zipped into the attached file. Attachment size is about 2.5 megabytes. nextcloud-desktop.valgrind.zip

[misch7]

Hey,

we've just released 2.6.1 RC1 which is built with Qt 5.12.5 on all platforms (also has OpenSSL 1.1.1d, so it features TLS 1.3).

You may give it a try: https://github.com/nextcloud/desktop/releases/tag/v2.6.1-rc1

Would be good to know if the problem was caused by the older Qt versions used in the previous builds.

[ConstantMown]

Hi,

I just updated to 2.6.1 RC1 using install package from https://download.nextcloud.com/desktop/prereleases/Mac/Nextcloud-qt5.12.5-2.6.1.20191018rc1.pkg

[ConstantMown]

Hi,

Unfortunately the nextcloud client process shows signs of memory leak.

2019-10-19 11:47 nextcloud 43.5 MB, 7 threads, ports 250 Nextcloud Extensions 16.5 MB, 3 threads, ports 148

2019-10-20 12:54 nextcloud 186.0 MB, 7 threads, ports 268 Nextcloud Extensions 17.0 MB, 3 threads, ports 149 Nextcloud Extensions 16.9 MB, 3 threads, ports 149

On the latter one there were two separate Nextcloud Extension processes.

In my case nextcloud is synchronizing the "nextCloud" folder which contains one 177 MB zip file other files two sub folders.

nextCloud/ 220 MB nextCloud/Documents 135 KB nextCloud/Photos 37 MB

[fuzunspm]

I'm having the same issue on macOS 10.15 and Nextcloud Client 2.5.3release (build 20190724)

Uptime: 5 days, 3 hours, 18 mins nextcloud memory = 3,30 GB -- Threads = 9

[msilvoli]

Hello,

Version 2.6.1 stable (build 20191104) running on Plasma 5 Opensuse Linux 15.1 still showing problems with memory management. 2 days after initial boot, client is consuming around 700MB + memory on swap file. No big sync with the server (all files less than 1MB) . I attached one photo of mem consumed and a more detailed report about the Nextcloud (apprun) memory usage.

nextcloud-client-2.6.1-memory-usage.txt

[sir-brickalot]

Same problem on a MacOS here: After a couple of days unattended run time I usually have around 13GB of memory used by Nextcloud desktop client on MacOS 10.14.6. File system APFS, 2 NC accounts, NC version 2.6.1 stable. Special situation might be that it's a hackintosh. Don't know if this matters.

I appended some screenshots and a log file in the hopes that might help. nextcloud.log.zip

[brianfmorris]

As others have mentioned this is a mac issue as well -

I am using Version 2.6.1stable (build 20191105) memory usage is 4.31GB of 24. I have only a few files in the cloud.

[nicokaiser]

Version 2.6.2rc1 (build 20191209) is still using ~300 MB after some days, with 667 threads.

There still must be some kind of memory leak, seems like 2.6.2rc1 just eats the memory at a slower rate. I will revert to ownCloud client (the one without the QtWebEngine bloatware) and see if it performs better over some days.

[nicokaiser]

With version 2.6.2 (release) it seems to stay quite stable at 7 threads / 70 MB now. I'll observe this further.

[ConstantMown]

I updated in MacOS 10.14.6 nextCloud desktop app to 2.6.2stable (build 20191224). The app was started on December 30th when its memory usage was 44,8 MB. After 48 hours the memory usage was 103,4 MB. Thread count has stayed same all the time 7 threads. IMHO some of the memory leaks were fixed in 2.6.2, but it seems to me that there are one or more memory leaks in 2.6.2. According to app previous activity was six days ago, when I marked one reminder as complete.

[VietTralala]

I also observe a huge memory consumption after a couple of minutes on the current AppImage Nextcloud 2.6.2-x86_46 obtainable via the website for Linux. My OS is Ubuntu 18.04.03. Right after Start: 76MB, 9 Threads After 10 min 1.3GB -1.8GB, >100 Threads

Threads are counted via: ps -o nlwp <pid-of-nextcloud>

EDIT: Nextcloud desktop client: Version 2.6.2stable (build 20191224) Built from Git revision 1d7455 on Dec 24 2019, 12:18:39 using Qt 5.12.5, OpenSSL 1.1.1d 10 Sep 2019

[msilvoli]

Plasma 5 Opensuse Linux 15.1 with client version 2.6.2stable (build 20191224) is much more stable compared with older versions like the comments of @nicokaiser and @ConstantMown. The memory consumption right after start was 41MB and I can see some memory management (going up and down) and almost no swap like my first comment here at this topic. I'll update this comment further with more details.

After 24h running : 82 MB / 10 Threads After 55h running : 170 MB / 10 Theads / 4.1GB disk Read / 2.4GB disk Write. After 73h running : 227 MB / 10 Theads / 4.1GB disk Read / 2.7GB disk Write. After 96h running : 228 MB / 10 Theads / 4.1GB disk Read / 2.7GB disk Write.

[FuturePilot]

2.6.2 on MacOS is definitely an improvement but there still seems to be something leaking. After 3 days it's only gone up to slightly over 300MB. Before it would be at 1GB or more after one day.

The odd thing is I am only seeing this problem on MacOS. On my Kubuntu 19.10 system it never goes above 100MB.

[VietTralala]

TLDR: I attached a valgrind log where I could observe nextcloud hogging memory.

This was my procedure: I build the nextcloud desktop client manually from source by following the wiki instructions

make .. -DCMAKE_INSTALL_PREFIX=~/.local/ -DCMAKE_BUILD_TYPE=Debug -DNO_SHIBBOLETH=1

The info from the running client: Built from Git revision 1d7455 on Jan 4 2020, 14:49:20 using Qt 5.9.5, OpenSSL 1.1.1 11 Sep 2018

I could still observe the memory growth to over 2GB, >200 Threads in around 24h of usage. Next I tried to run valgrind on the compiled binary ~/.local/bin/nextcloud following these instructions. The command was:

G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind -v --tool=memcheck --leak-check=full --num-callers=40 --log-file=valgrind.log ./nextcloud

---

I've let nextcloud run for a couple of hours until the memory growth was above 2GB,>200Threads were running and then manually exited the nextcloud client. The resulting log file is attached:

valgrind.log

I think the interesting part LEAK SUMMARY begins at line 6933. However I miss the expertise to make any sense of it. But if anyone wants more data to debug this issue I am wiling to help.

[ConstantMown]

My Nextcloud client, version 2.6.2stable (build 20191224) in macOS 10.14.6, has been up for fifteen days. The memory usage according to Activity Monitor is 333 MB and 7 threads. Stopping Nextcloud client and killing Nextcloud Extensions manually and then starting nextcloud client, its memory usage is 44 MB.