microsoft
Help wanted: A question about mi_track_malloc
I am implementing a custom memory tracker to monitor allocations and deallocations from the mimalloc memory allocator. While I am aware that tools like Valgrind exist, I have opted to develop my own for a more lightweight solution. However, I have encountered a challenge that I hope the community can help me resolve, and I would be immensely grateful for any assistance.
Description
In a system with limited RAM, address reuse is an inevitable phenomenon, especially in memory pool implementations. This means that a given tuple (p, size) can correspond to multiple distinct allocation events over time (we can distinguish them as "different allocations" by capturing a call stack). The difficulty arises during the deallocation phase.
Specifically, since mi_track_free_size function only accepts the pointer p and the size as parameters, it becomes impossible to determine which of the previous allocations, associated with their respective call stacks, this particular (p, size) tuple corresponds to.
What I am doing
My implementation is inserting macro defs: below this: https://github.com/microsoft/mimalloc/blob/09a27098aa6e9286518bd9c74e6ffa7199c3f04e/include/mimalloc/track.h#L89
#elif MY_TRACK_ENABLED
#define MI_TRACK_ENABLED 1
#define MI_TRACK_HEAP_DESTROY 1
#define MI_TRACK_TOOL "MY_TRACE_TOOL"
#include <mytrack/my-hooks.hh>
#define mi_track_init() my_track_init();
#define mi_track_malloc_size(p,r,s,z) my_track_malloc(p,r,s,z)
#define mi_track_free_size(p,size) my_track_free(p,size)
#else
other
- code version I am using:
release 3.0.9 - I have ensured that mimalloc has linked mytrack dynamically, and soved reentrant recursion problems.
- if my implementation is not the right way, please point out
huge thanks.
Is there any way that I could add extra bytes before the pointer to store, like, a unique allocation id?
While I can't provide you with a comprehensive answer, I wanted to point out that:
In a system with limited RAM, address reuse is an inevitable phenomenon
Technically, it's not the RAM that's limiting, it's "just" the virtual address space - which is much larger, and if it's exhaustively used, the frequency of address reuse would be reduced.
To use more of the virtual address space, it would perhaps require something like not reusing virtual memory allocations from the OS, but rather to favor requesting new allocations. But generally, I have no idea how to go about this. (But I have a feeling this would have an impact on performance any may cause increased memory usage...)
Another observation...
it becomes impossible to determine which of the previous allocations, associated with their respective call stacks, this particular
(p, size)tuple corresponds to.
Actually, if allocations and deallocations are properly matched, this will always match to the last allocation at that address. Of course, this won't help you if you want to actually track down allocation and deallocation mismatches (ie leaks and double-frees). ¯\_(ツ)_/¯