jemallocator icon indicating copy to clipboard operation
jemallocator copied to clipboard

Failed to increase stack memory

Open dertin opened this issue 2 years ago • 4 comments

Why does increasing the stack memory cause an error when creating the background thread?

tikv-jemallocator = {version = "0.5", features = ["background_threads"] }

ulimit -s 536870912
./my-program

result: <jemalloc>: arena 0 background thread creation failed (11)

dertin avatar Feb 03 '23 12:02 dertin

Maybe it's just too large (512MiB) for the thread to be spawn.

BusyJay avatar Feb 06 '23 08:02 BusyJay

It only gives a problem when we have activated the features = ["background_threads"]

I tried with values of stack memory lower than 512MB and the problem is present when setting any value outside the default.

ulimit -s <bytes> this should increase the stack memory over the main function.

I tried to increase the stack memory for threads but it didn't fix the problem. RUSTFLAGS="-C link-args=-Wl,-zstack-size=<bytes>" RUST_MIN_STACK=<bytes>

What is this behavior due to?

dertin avatar Feb 06 '23 13:02 dertin

When enabling background_threads, background threads are created to do purging in the background. The error code 11 is "Resource temporarily unavailable" and returned when spawning the thread in C. Not look into the code yet, but I think it's probably due to 512MB is not a good value.

BusyJay avatar Feb 06 '23 14:02 BusyJay

There should be no problem with thread creation if there is more stack memory for the main function.

Higher stack memory values are used for applications with high recursion, among other rare cases. I think it would be good to find the cause to better understand the operation of what is happening in the back.

dertin avatar Feb 08 '23 18:02 dertin