High (8GB) memory allocation when running with a high RLIMIT_NOFILE

[thomassteinreiter]

Steps to reproduce:

1. Download btdu-static-x86_64 v0.6.0.
2. Mount a Btrfs filesystem (e.g., sudo mount -o subvolid=5 /dev/sda3 /mnt/btrfs-root on Fedora 42).
3. Reproduce with high limit:
   • Run: sudo prlimit --nofile=1073741816:1073741816 -- ./btdu-static-x86_64 /mnt/btrfs-root
4. Monitor with top—memory spikes to 8GiB+; process hangs, even before any TUI is shown

Why does this matter? I found this behaviour when trying to set up a (privileged) docker devcontainer, that is able to call btdu (for a test automation usecase). Inside the docker container, the default RLIMIT_NOFILE is very high (~1e9) which then seems to cause a high memory allocation in the btdu child process

My current workaround is ulimit inside the devcontainer (ulimit -n 4096' >> ~/.bashrc) Then, everything seems to run normally again.

Here is an strace of the reproduction: strace.txt

these two lines seems to be the interesting part:

[pid 261781] prlimit64(0, RLIMIT_NOFILE, NULL, {rlim_cur=1073741816, rlim_max=1073741816}) = 0
[pid 261781] mmap(NULL, 8589934592, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe158a00000

The process first calls prlimit64 to get the current file descriptor limit, which returns a very high value (rlim_cur=1073741816) Immediately after, it allocates 8589934592 bytes via mmap

Maybe it is possible to put a hard cap on the resources that btdu is using, when having a high RLIMIT_NOFILE.