Large allocation fails to find free space in preallocated arena

[jprendes]

I am trying to use mimalloc in memory constrained environments, so I want to use a preallocated arena. It is similar to #1115 but it happens with mimaloc v2 (from main), and for larger allocations (~60MiB or more). Also, it seems to ignore the disallow_os_alloc option, and tries to allocate memory from the OS before failing to allocate.

Here's a small reproducer

#include <stdio.h>
#include <mimalloc.h>

char memory[256 * 1024 * 1024] = {0};

int main(void) {
  mi_option_set_enabled(mi_option_disallow_os_alloc, true);
  mi_option_set_enabled(mi_option_verbose, true);

  mi_manage_os_memory(
    memory,
    sizeof(memory),
    1 /* committed */,
    0 /* large     */,
    0 /* zero      */,
    -1 /* numa_node */
  );

  void * ptr = mi_malloc(64 * 1024 * 1024);

  fprintf(stderr, "ptr = %p\n", ptr);

  mi_option_set_enabled(mi_option_verbose, false);
  return ptr ? 0 : 1;
}

This produces the following output

mimalloc: using 1 numa regions
mimalloc: reserved 1048576 KiB memory
mimalloc: reserved 1048576 KiB memory
mimalloc: error: unable to allocate memory (67108864 bytes)
ptr = (nil)

While I would expect it to produce something like

mimalloc: using 1 numa regions
ptr = 0x55e8ee010000

[jprendes]

From what I've seen, it seems to happen when the allocation would require more than 2 bits from the bitmap. For smaller allocations (that require less that 2 bits), _mi_bitmap_try_find_from_claim_across takes a different path

  if (count <= 2) {
    // we don't bother with crossover fields for small counts
    return _mi_bitmap_try_find_from_claim(bitmap, bitmap_fields, start_field_idx, count, bitmap_idx);
  }

In the path taken when the allocation (with it's metadata overhead) takes more than 2 bits, there's a commented out block of code:

    // first try to claim inside a field
    /*
    if (count <= MI_BITMAP_FIELD_BITS) {
      if (_mi_bitmap_try_find_claim_field(bitmap, idx, count, bitmap_idx)) {
        return true;
      }
    }
    */

I think the issue is related to that commented out block, which was changed in 027b22aaf25e3f76028d149a1b2a50ab92286699 If I uncomment that block, the allocation succeeds.