Large allocations (>= 32 Mb) ignore reserved OS memory

[a-sid]

Hi. Thank you for this excellent project.

We noticed that large allocations are not served from preallocated arenas in mimalloc 3 (at least, in 3.0.3). Consider a simple repro example:

$ cat repro.cpp

#include <vector>

__attribute__((noinline)) std::vector<char> CreateLargeBuffer() {
  std::vector<char> buffer(1024ull * 1024 * 32);  // Allocate 32 Mb from heap.
  return buffer;
}

int main() {
  for (int i = 0; i < 100; ++i) {  // Allocate memory 100 times.
    auto buffer = CreateLargeBuffer();
    (void)buffer;  // Prevent unused variable warning.
  }
  return 0;
}

# Trace mmap call stacks for the program run.
$ MIMALLOC_RESERVE_OS_MEMORY=10GB strace -k -f -e mmap,munmap -o log ./repro
$ grep 'CreateLarge' log | wc -l
200  # 100 mmap and munmap pairs.

We see that despite the 10 GB arena preallocated with MIMALLOC_RESERVE_OS_MEMORY=10GB every allocation call still ends up in mmap and munmap call pair. This was quite unexpected. Arena preallocation was recommended in another issue: https://github.com/microsoft/mimalloc/issues/447. But it looks like this workaround doesn't work anymore. Does someone have any suggestion about preventing mapping/unmapping memory? Is it a regression or somewhat expected?

[a-sid]

I've checked 3.1.5, and the result is the same.

[daanx]

Ah yes -- that is a "regression" in v3 and should be fixed. Hopefully we can get that into the next release :-)