jemalloc failure on raspberry pi 5

[nickkaltner]

Describe the bug Running on a raspberry pi 5, there are 16KB page sizes which breaks the jemalloc configuration on quickwit

It won't run, here is the output;

./quickwit-v0.8.0/quickwit --help
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
memory allocation of 5 bytes failed
Aborted

confirmation of the page size;

$ getconf PAGE_SIZE
16384

Steps to reproduce (if applicable) Steps to reproduce the behavior:

1. follow the installation instructions on a raspberry pi 5
2. watch it crash

Expected behavior

It should run as per the instructions

Configuration: Please provide:

1. Output of quickwit --version

n/a because this crashes but the version i was running is 0.8.0 on aarch64-unknown-linux-gnu (7afa596)

3. The index_config.yaml

n/a

[nickkaltner]

note there is a temporary workaround which is adding the line kernel=kernel8.img to /boot/firmware/config.txt and rebooting.

you can confirm the fix has applied by

$ getconf PAGE_SIZE
4096

and then

./quickwit --version
Quickwit 0.8.0 (aarch64-unknown-linux-gnu 2024-03-18T15:20:23Z 7af4596)

[trinity-1686a]

dumping some observations:

• cross-compiling with JEMALLOC_SYS_WITH_LG_PAGE=16 should make things work on a 16k page system, and not break on 4k according to https://github.com/home-assistant/docker-base/pull/248. (we probably want 16 and not 14 because BCM2712 reportedly default to 64k pages). I don't know the exact tradeoff, but i guess this would cause jemalloc to allocate slightly more than required
• cross-compiling without --features jemalloc (which is implied by --features release-feature-set) would use system allocator and not cause issue. In my experience, GNU allocator tend to bloat a lot with rust, so targetting musl instead of gnu might be a decent idea too.
• compiling directly on a system with PAGE_SIZE=16384 should also work as jemalloc auto-detect host's configuration.

[fulmicoton]

A simpler alternative would be to compile without jemalloc. It is an optional feature of quickwit.

[tiagozip]

Any update on this please? I would definitely prefer to not have to compile everything without jemalloc or change my system's page size.