Memory issue

[daiyam]

Hi @basharovV,

There is a memory usage issue with the CanvasLibraryView component.

With nearly 24K songs, on the album view:

• at boot, the tauri process takes 2.1GB of memory
• after a minute, it goes down to 1.2GB
• with the component disabled, 180MB

With 350 songs, it goes from 400MB to 100MB...

I will try to see where exactly is the problem...

[basharovV]

That ain't good, I'll take a look as well.

[daiyam]

Not sure what's going on, I can't replicate the issue with the same code, still in the same session... weird!

I will look if it happens again...

[daiyam]

OK, this is weird! I'm getting the issue again...

[daiyam]

I've tried to fix the issue by updating tauri but it didn't fix it.

The process which eats the memory, is the WebKit's WebContent process launched by tauri. Now, I'm logging its usage to see what's going on...

[daiyam]

I think I found the issue...

I've added the following code:

navigator.storage.estimate().then(({ usage, quota }) => {
    console.warn(`Database size: ${usage} bytes`);
});

I gave me: Database size: 1_822_589_588 bytes (_ were manually added) I have 41K songs and 2.6K albums

I've finally found the sqlite file at ~/Library/WebKit/Musicat/WebsiteData/Default. 1.82GB...

So, let's say 30K per song. Its JSON is under 1K. There is an issue there.

[daiyam]

Ok, the real issue is here: https://github.com/basharovV/musicat/blob/4b0807842fc01ffdb1e0954066c9f4ce4fd6ed0b/src/lib/views/CanvasLibraryView.svelte#L139

It loads the whole 41K songs in memory (in about 5.8s).

It should load only a part of those songs (maybe 1K) and move the part/window based on the scrolling. Some filtering and sorting would need to be moved to the database.

The database' size is an issue (1.82GB is way to big for 41K songs) but it isn't the main issue for the memory usage.

[daiyam]

Not sure what's going on with that database... I've re-importing the songs in dev mode, only 250Mo. I moved the db file to the release app and:

• the loading is way faster
• only 500Mo memory usage

I've look at the big db, in the dev tools (storage tab), I didn't see anything odd.

[basharovV]

It should load only a part of those songs (maybe 1K) and move the part/window based on the scrolling. Some filtering and sorting would need to be moved to the database.

I tried to implement this via .offset() and limit(), but it's not quite as performant. Will continue to investigate

results = db.songs
    .orderBy(
        $query.orderBy === "artist"
            ? "[artist+year+album+trackNumber]"
            : $query.orderBy === "album"
              ? "[album+trackNumber]"
              : $query.orderBy,
    )
    .offset($libraryPage.offset) // pagination
    .limit($libraryPage.size);