mmtk
Accounting VM-specific allocations
This allows the VM to add arbitrary VM-specific allocations into the heap size.
I am confused by this, and why we would want this, e.g. what problem is PR is trying to solve.
We had previously discussed the need to support runtime allocations (eg by the binding and even by MMTk core via rust), and have their footprint included in our heap size calculations.
The approach we discussed was to have a malloc space (or malloc spaces) into which we could allocate. The accounting for this space is then (automatically) identical to the way any other space is accounted, although the memory within the space is explicitly managed rather than garbage collected.
This could even be used as the replacement allocator for Rust, giving us automatic accounting for Rust data structures that we allocate (for example, for remembered sets, work queues etc).
In the short term we can do this relatively straightforwardly by using our existing malloc support (which calls through to standard library malloc implementations). In the medium term we should be supporting this with our own native implementation of mimalloc (which I believe is close to being ready).
So.... I am confused by this PR, and worried about what it is doing to the API, and how it fits in with the above.
We have APIs to allow VMs to do malloc with MMTk (and count that into MMTk's heap). This PR allows VM to manage some allocation by themselves, but also count these allocation into the heap size.
The direct motivation from Kunshan is that for Ruby, there are some allocation before Ruby even initializes MMTk (they cannot call MMTk API before initializing MMTk) -- for example, Ruby allocates its own VM and the main thread before it initializes MMTk. So the PR allows Ruby to allocate those data structures with its own allocator/malloc, and later when MMTk is initialized, Ruby can inform us of those allocation.
I think this could also be generally useful when a VM would like to do part of the allocation themselves, but also want to count those allocation into the heap size. It is overly restrictive for us to say that if a VM wants to use MMTk, they have to delegate all the allocation to MMTk.
@steveblackburn I didn't mean to replace the counted malloc API we provide, but provide another option.
There are several reasons for this.
The first reason is what Yi just mentioned. In Ruby, there is a cyclic dependency. It needs to malloc the Ruby VM itself rb_vm_t and its initial thread rb_thread_t before the heap is allocated, at which time we cannot call any MMTk API except instantiating MMTk. Actually, Ruby has its own accounted malloc (ruby_xmalloc and ruby_xfree). It requires Ruby's own GC to be initialized, too, so Ruby couldn't use ruby_xmalloc to allocate rb_vm_t, eithre. To mitigate the problem, Ruby introduced a function ruby_mimmalloc ("mim" means "mimic", not to be confused with "mimalloc") that does plain malloc without accounting, and expect it to be "compatible with" ruby_xfree. Since in Ruby, both ruby_mimmalloc and ruby_xfree use malloc and free underneath, it is actually compatible. But I can't replace ruby_xfree with our mmtk_free_with_size and expect it to be compatible with plain malloc. Also note that pairing ruby_mimmalloc with ruby_xfree makes the accounting imprecise (It does not increase the counter, but it decreases the counter). Ruby mitigates this by letting the decrement "saturate" at (and do not go below) 0.
The second reason is that some VMs are already doing the acccounting. Like I said before, Ruby already counts its xmalloc/xfree, and it is nice if we provide an API so we can directly bridge with theirs.
The third reason is that the VM may expect the memory to be allocated in certain ways. It may require a malloc library that has certain capabilities, or not using malloc at all. For example,
- Ruby expects the existence of
malloc_usable_size(It is a GNU extension available on Linux. See the man page:man malloc_usable_size) which returns the size of themalloc-ed memory. MMTk's API doesn't provide an equivalent for this, and I have to hide a size field in front of everymalloc-ed memory so that it can be passed to MMTk'sfree_with_sizeAPI (Ruby's size hint passed toxfreeis usually unavailable). - SpiderMonkey and V8 are known to use "arena" allocator for their JIT compilers. It allocates memory using bump-pointer, but frees the entire arena when a job completes.
- They may use
mmapif they have to, and they may count that as part of the heap size.
The fourth reason is that some VMs provide API for its users to count off-heap allocations. Android is one example. IIRC, Android, starting from version "Oreo", moved the storage of bitmaps (graphic images, pictures) off heap, but count them as part of the heap size so it can trigger GC promptly. They do this by providing an API to C programmers so that they can report their allocations to the VM. (See Heap::RegisterNativeAllocation in https://android.googlesource.com/platform/art/+/refs/heads/master/runtime/gc/heap.cc#4147) If the VM provides such a mechanism to user libraries (or its huge standard library), the libraries will be beyond the control of the VM developers, and even the VM cannot control how their libraries allocate memory. Although replacing malloc is one way, it is not general enough to cover all cases.
So although providing our own counted malloc API does have its advantages (such as counting all malloc calls, and using our own implementation which could be more efficient), there are still cases where letting the user count their own allocation can be useful, and sometimes required by the VM (in the case the VM provides such an API to its users).
OK.
I understand the desire to account for off-heap memory as part of the heap. However, I vigorously disagree with the premise of using off-heap memory in the first instance. As far as I am aware it is a terrible hack to get around poor GC design decisions. This is something that MMTk is designed to address not support.
Overall, I am rather worried about a lot of this. I suggest we have a review of this and other (existing?) changes to the API.
More substantially, this is not how we account for memory, and not how any memory manager should account for memory. We very carefully account for memory in terms of pages in use, not in terms of bytes allocated.
As we discussed in today's meeting, there may be an alternative solution.
We provide an API that simply asks the VM how much memory it wants to count into the heap size. mmtk-core will simply add that number into the heap size when determining whether to trigger GC.
The API can also ask the VM about more concrete GC advices, such as "The used memory in MMTk spaces is currently X pages. Do you want to do a GC now?" and the VM may answer "Yes. Please.", or "No. Please don't do GC now.", or "I don't know. You decide."
I'll try adding a API to ask Ruby for its recorded xmalloc-ed size in another PR.