mimalloc icon indicating copy to clipboard operation
mimalloc copied to clipboard

How do adjust alignment and segment size and/or page size?

Open FlogramMatt opened this issue 2 years ago • 1 comments

Trying to get mimalloc working with wasm, I've seen this issue: https://github.com/microsoft/mimalloc/issues/308

My understanding is this suggestion means we need to allocate RAM in advance. For my application, I can't afford to allocate 1GB ram just in case we need it, we might have multiple versions running in parallel and it just doesn't work for the customer. Additionally, note that with emscripten, can use 'TOTAL_MEMORY'(see Makefile) to set the maximum size the memory can grow to without having to also allocate it up front(my understanding anyway).

I tried to compile using emscripten and it's finally compiling but unfortunately the first time it allocates memory it calls _mi_os_prim_alloc_aligned with an size: 33554432 and alignment: 4194304. Which I believe comes from MI_SEGMENT_ALIGN in types.h?

It would be nice if alignment was more like 64 bytes or less, not 4 MB.. right? Additionally, I'd like to adjust how much memory it tries to allocate at the same time, getting it down to about 1 MB at a time unless more is needed for the current allocation.

(You can see my code changes here: https://github.com/microsoft/mimalloc/compare/master...Flogram:mimalloc_wasm:master )

If you install emscripten, can run this using 'make' and usually should call 'make clean' before recalling 'make'. Then use 'http-server' from npm, found here: https://www.npmjs.com/package/http-server

Here is my own homemade stack trace in console.log to figure out what's going on:

_mi_heap_malloc_zero_ex size: 10 
_mi_heap_malloc_zero_ex MI_SMALL_SIZE_MAX: 512 
heap is: 93376 
size + MI_PADDING_SIZE is: 18 
_mi_heap_get_free_small_page returned: 84160 
_mi_page_malloc called. size: 18 
_mi_page_malloc block value before allocation: 0 
_mi_malloc_generic called with size:  18 , zero:  0 , huge_alignment:  0 
_mi_malloc_generic is heap initialized:  1 
mi_find_page heap  93376 
mi_find_page size  18 
mi_find_page huge_alignment  0 
mi_find_page req_size  10 
mi_reserve_os_memory_ex called _mi_os_alloc_aligned 1 
  size:  33554432 
  alignment(MI_SEGMENT_ALIGN):  0 
_mi_os_prim_alloc_aligned called. 
  size:  33554432 
  alignment:  4194304 
mi_os_prim_alloc calling mi_os_prim_alloc_aligned. 
  size:  33554432 
  alignment:  4194304 
mi_os_prim_alloc_aligned called. 
  size:  33554432 
  alignment:  4194304 
mi_os_prim_alloc_aligned called mi_os_prim_alloc 1. alignment:  4194304 
mi_os_prim_alloc called. 
  size:  33554432 
  try_alignment:  4194304 
_mi_prim_alloc called. 
  size:  33554432 
  try_alignment:  4194304 
_mi_prim_mem_grow called by _mi_prim_alloc. 
_mi_prim_mem_grow 
  size:  33554432 
  try_alignment:  4194304 
_mi_prim_mem_grow called wasm version 
  size:  0 
  _mi_os_page_size(): 65536 
  _mi_divide_up(size, _mi_os_page_size()): 0 
  __builtin_wasm_memory_size(0): 256 
mi_prim_mem_grow called mi_memory_grow 
  current:  16777216 
_mi_prim_mem_grow called wasm version 
  size:  33554432 
  _mi_os_page_size(): 65536 
  _mi_divide_up(size, _mi_os_page_size()): 512 
  __builtin_wasm_memory_size(0): 256 
mi_prim_mem_grow called mi_memory_grow 
  base:  0 
mi_prim_mem_grow returning 
  p:  0 
after _mi_prim_mem_grow 
  *addr:  0 
mimalloc: warning: 
unable to allocate OS memory (error: 48 (0x30), size: 0x2000000 bytes, align: 0x400000, commit: 0, allow large: 0)

Thank you!

FlogramMatt avatar Jul 08 '23 23:07 FlogramMatt

I never heard a response to this, looking into it again. I would appreciate help still. Thanks.

FlogramMatt avatar Nov 25 '23 19:11 FlogramMatt