Memoryleak after long system run

[solisinvictum]

Hey,

my System:

[user@1950x ~]$ uname -a
Linux 1950x 6.2.9-1-cachyos #1 SMP PREEMPT_DYNAMIC Thu, 30 Mar 2023 18:47:21 +0000 x86_64 GNU/Linux
[user@1950x ~]$ 
[user@1950x ~]$ neofetch
           .-------------------------:                   user@1950x 
          .+=========================.                   ---------- 
         :++===++==================-       :++-          OS: CachyOS x86_64 
        :*++====+++++=============-        .==:          Host: Micro-Star International Co., Ltd. MEG X399 CREATION (MS-7B92) 
       -*+++=====+***++==========:                       Kernel: 6.2.9-1-cachyos 
      =*++++========------------:                        Uptime: 8 hours, 55 mins 
     =*+++++=====-                     ...               Packages: 1830 (pacman) 
   .+*+++++=-===:                    .=+++=:             Shell: bash 5.1.16 
  :++++=====-==:                     -*****+             Resolution: 1920x1080, 1920x1080, 3840x2160, 3840x2160 
 :++========-=.                      .=+**+.             DE: Plasma 5.27.3 
.+==========-.                          .                WM: KWin 
 :+++++++====-                                .--==-.    WM Theme: Breeze 
  :++==========.                             :+++++++:   Theme: CachyOSNord [Plasma], Breeze [GTK2/3] 
   .-===========.                            =*****+*+   Icons: [Plasma], breeze-dark [GTK2/3] 
    .-===========:                           .+*****+:   Terminal: konsole 
      -=======++++:::::::::::::::::::::::::-:  .---:     CPU: AMD Ryzen Threadripper 1950X (32) @ 3.400GHz 
       :======++++====+++******************=.            GPU: NVIDIA GeForce GTX 1080 Ti 
        :=====+++==========++++++++++++++*-              GPU: NVIDIA GeForce GTX 1080 Ti 
         .====++==============++++++++++*-               Memory: 7044MiB / 64292MiB 
          .===+==================+++++++:
           .-=======================+++:                                         
             ..........................                                          

[user@1950x ~]$ 

After the System runs longer as 8 Hours, the Memory get full, until the System crash. It seems Fsearch is the cause of it:

[user@1950x ~]$ ./test.sh 
MEMORY                Top 10 processes        SWAP
49993.07M fsearch                              4841.00M fsearch
 1300.32M librewolf/librewolf                  2418.75M fsearch
 1117.36M electron/electron                    2092.75M fsearch
  474.18M plasmashell                          1958.00M fsearch
  381.33M kwin_x11                             1315.50M clamd
  302.27M mpv                                   992.75M fsearch
  279.82M betterbird                            984.25M fsearch
  178.78M Xorg                                  248.50M fsearch
  147.43M fancontrol_gui                        200.25M plasmashell
  125.37M dolphin                               178.00M fsearch

   16042M SwapTotal
[user@1950x ~]$

It use over 50GB of Ram on my System (System have 64gb), and its growing with no limit.

It seems that this happen, because hundreds of fsearch --update-database threads get started.

For now, i disabled that cron job to update the database.

[cboxdoerfer]

Thx for the detailed report. This sounds like a really bad bug.

Can you provide some additional information: How frequently does the cron job run? How many entries does the database have on average? And how long does a single database update rougly take?

[solisinvictum]

How frequently does the cron job run?

*/15 * * * * fsearch --update-database >/dev/null 2>&1

How many entries does the database have on average?

Fsearch says XXXX (as soon fsearch have done his indexing, i edit it)

And how long does a single database update rougly take?

Dont know. Because i had only the "background service" using for it. Started it right now (gui) and it is indexing, seems to to run atleast for a hour or longer.

[solisinvictum]

i make a new post, because to underline this.

it is still indexing. i have over 172 timeshift backups. and the indexing is at 18.02.2023. so weeks left.

i could imagine, that the fsearch --update-databse is ignoring that already one indexing job is running, and starts another. and 15 minutes later another one, and again after 15 minutes later again and so forth. until all ram is used.

but so far i remember, in the wiki or so is written, that fsearch checks ifs already running a database update?

that would explain, why hundreds of threads getting started.

Edit:

Index from GUI is finished. 149.540.116 Entries.

[cboxdoerfer]

Thank you, that helps a lot.

i could imagine, that the fsearch --update-databse is ignoring that already one indexing job is running, and starts another. and 15 minutes later another one, and again after 15 minutes later again and so forth. until all ram is used.

Yes, that's what I suspected as well.

but so far i remember, in the wiki or so is written, that fsearch checks ifs already running a database update?

Yes, it does check if an update is taking place but it should queue a new update which should run immediately after the current one finishes. But it's not supposed to have more than one update in the queue. However it seems there's something not working right, because there should never be hundreds of threads or such a high RAM usage.

Index from GUI is finished. 149.540.116 Entries.

That's quite a lot. You should expect FSearch to use roughly 20GB of RAM with that many entries in the database. It can also be a bit more depending on how long your file names are on average. During a database update and while the app is running however the RAM usage is also expected to grow to twice it's sice for a certain amount of time, because FSearch needs to keep the old and new building database in RAM at the same time. So in your case it's expected that the RAM usage can grow to something like 40GB while a scan is running and FSearch is in use.

If that's an issue for you you might want to exclude some unnecessary folders and files from the database. The next release will also allow you to reduce RAM usage, by excluding certain properties or fast sort orders (file size, modification time, ...) from being indexed.

But there's definitely some bug going on here and I'll see if I can reproduce it.