mujx
High memory usage?
Hi again,
After building from master again and running strip on the binary to get it down to 7.1 MB, I wondered if it would use less memory. But looking at the nheko process now, sitting in 6 rooms, only 2 of which have had any event history loaded since starting the program, the RSS is at 220 MB, with 53.8 MB shared, which seems high.
Is there something about the Qt5 controls that nheko uses that's causing this, or is there a bug? 220 MB (or even 170) just seems excessive for how much actual data should currently be resident in the program.
Thanks.
Running nightly from today gives me
on my @Half-Shot:half-shot.uk account. Which contains about 276 joined rooms.
Ok, I ran with massif:
-
valgrind --tool=massif nheko - Waited for window to appear.
- Clicked on each room in the room list.
- Tray icon->Quit.
Here's the output: massif.out.txt
I think the working theory is that this is caused by creating widgets for every single event, rather than just the ones in view in the room you're currently looking at.
Here's the output: massif.out.txt
Using massif visualizer you can see that 52 MB are the user avatars 67MB the state events. Peak memory usage was 162MB.
One optimization could be to store the user avatars in cache.
I think the working theory is that this is caused by creating widgets for every single event, rather than just the ones in view in the room you're currently looking at.
This has to be tested that is not affecting performance (by removing and restoring widgets). Also there are other issues like remembering scrollbar position etc. But probably that's the best bet to keep memory low.
I have account with 150+ rooms (see also mujx/nheko#89) and I have pretty same results as what I have with Riot (except CPU usage where nheko is better):
I think it's that so because avatar of hundreds of rooms with thousands of users all stored in RAM uncompressed. Maybe some algorithm could be implemented to compress loaded avatars (and load big ones when you look at avatar closer, zooming it) and unload cached avatars from rooms which weren't visited in last N minutes, something like that. Am I right?
After next run, without initial sync, it eats less than 500 MiB:
Could it be memory leak? Or is it just reserved free memory by OS?
@unclechu The avatars are downloaded on demand (i.e only those that are visible when you open a room). I am still investigating to find out exactly what data are responsible for such high memory.
There are some improvements regarding the memory usage currently in the master branch. Let me now if those updates bring memory usage back to normal levels.
@mujx I tried to build it by make release but it failed, see last parts of log:
nheko-2018-04-22-1642f3c-build-error.log
@unclechu This is issue #298. Check the readme for the supported compilers. You can grab an nightly appimage just to test this out.
After first start (setting up cache was mentioned in log) of nighlty release (nheko-862.nightly-1.x86_64.rpm) it ate a lot (I have 150+ rooms):
But after restart it's okay:
As I remember I was having about 250MiB approximately, not it's about 135MiB.
I have a problem with nheko running in the system tray for a long time. Then the window want show up and needs a restart.
@notTito Is it related to high memory usage? If not open a new issue with some info about the version you're using and your system.
I will try. Thing is it takes time to reproduce but it happens reliably. OS is Manjaro Linux, nheko build φrom AUR.
3.3 GiB after initial sync with 0.4.0 from repos of Fedora 28 Workstation.
And about 150 MiB after second run (to compare Riot in Firefox 59 usually eats about 2 GiB).
I caught it do the same but memory usage is normal 50mb, but all I see is a window decoration with an empty content.
@notTito it depends on how many rooms do you have. In my case it's about 150 rooms.
So during initial sync the huge json has to be parsed into a json type and then copied again to a strongly typed representation.
That will cause a spike in memory but on Linux that reserved memory might not be released back to the OS. That doesn't mean that nheko still uses that. After a restart, memory appears to be normal.
On the other hand on macOS (and Windows probably?) the memory is released immediately so it won't be noticeable.
Some solutions:
- Use a more memory-efficient json library or struct representation.
- Protocol improvements to lazy load data.
On the other hand on macOS (and Windows probably?) the memory is released immediately so it won't be noticeable.
Why would that be the case, compared to Linux? If that were so, it would seem that Linux systems would be generally short of memory compared to macOS/Windows systems for the same tasks.
Why would that be the case, compared to Linux? If that were so, it would seem that Linux systems would be generally short of memory compared to macOS/Windows systems for the same tasks.
It's up to the memory allocator on each system to decide how the memory is managed. This doesn't mean that memory will always be reserved.
From https://www.gnu.org/software/libc/manual/html_node/Freeing-after-Malloc.html#Freeing-after-Malloc
Occasionally, free can actually return memory to the operating system and make the process smaller. Usually, all it can do is allow a later call to malloc to reuse the space. In the meantime, the space remains in your program as part of a free-list used internally by malloc.