microsoft
mimalloc-override.h does not work correctly with namespaces
Hello! Recently, I decided to build a project with Clang using mimalloc. I opted for overriding through header files, as I believe this is the most reliable and controllable way to override calls. The flags are quite simple for this:
C*FLAGS="-include mimalloc-override.h"
LDFLAGS="-lmimalloc"
However, Clang failed to compile with errors
/usr/lib/llvm/21/include/llvm/Support/MemAlloc.h:53:18: error: no member named 'mi_realloc' in namespace 'std'; did you mean simply 'mi_realloc'?
53 | void *Result = std::realloc(Ptr, Sz);
| ^ ~~~~~~~~~~~~~~~~
/usr/include/mimalloc.h:111:40: note: 'mi_realloc' declared here
It seems that the method of overriding using #define does not account for calls that may use namespaces (or their aliases). For now, one option I see is to add definitions of mi_realloc into the std namespace, creating something like this:
mimalloc-override.h
#include <mimalloc.h>
#define malloc(n) mi_malloc(n)
#if defined(__cplusplus)
namespace std {
inline auto mi_malloc(size_t n) { return ::mi_malloc(n); }
}
#endif
out.cpp
#include "mimalloc-override.h"
#include <cstdlib>
int main() {
auto _ = std::malloc(8);
}
preprocessed_out.cpp
# 2 "./mimalloc-override.h" 2
namespace std {
inline auto mi_malloc(size_t n) { return ::mi_malloc(n); }
}
# 2 "out.cpp" 2
int main() {
auto _ = std::mi_malloc(8);
}
Thank you for your time and for the great project! :)
I did some digging in the header file mimalloc-override.h, and the #define started conflicting not only with namespaces but also with any objects that contain free, malloc, and so on. So far, I've resorted to using the interpose project to replace functions at runtime. It works great, but I have to set the --allow-multiple-definition flag because multiple implementations of the same __interpose_ are created. However, in its current format, it works well, and the compiler also handles such declarations nicely. The overhead is just one call to the function without PLT, and it turned out like this:
/* ----------------------------------------------------------------------------
Copyright (c) 2018-2020 Microsoft Research, Daan Leijen
This is free software; you can redistribute it and/or modify it under the
terms of the MIT license. A copy of the license can be found in the file
"LICENSE" at the root of this distribution.
-----------------------------------------------------------------------------*/
#pragma once
#ifndef MIMALLOC_OVERRIDE_H
#define MIMALLOC_OVERRIDE_H
#ifndef _GNU_SOURCE
#define _GNU_SOURCE
#endif
/* ----------------------------------------------------------------------------
This header can be used to statically redirect malloc/free and new/delete
to the mimalloc variants. This can be useful if one can include this file on
each source file in a project (but be careful when using external code to
not accidentally mix pointers from different allocators).
-----------------------------------------------------------------------------*/
#include <mimalloc.h>
#include <stdlib.h>
#include <malloc.h>
#if !defined(__cplusplus)
#include <interpose.h>
// Standard C allocation
INTERPOSE_C(void*, malloc, (size_t n), (n)) { return mi_malloc(n); }
INTERPOSE_C(void*, calloc, (size_t n, size_t c), (n, c)) { return mi_calloc(n, c); }
INTERPOSE_C(void*, realloc, (void* p, size_t n), (p, n)) { return mi_realloc(p, n); }
INTERPOSE_C_VOID(free, (void* p), (p)) { return mi_free(p); }
INTERPOSE_C(char*, strdup, (const char* s), (s)) { return mi_strdup(s); }
INTERPOSE_C(char*, strndup, (const char* s, size_t n), (s, n)) { return mi_strndup(s, n); }
INTERPOSE_C(char*, realpath, (const char* f, char* n), (f, n)) { return mi_realpath(f, n); }
INTERPOSE_C(size_t, malloc_usable_size, (void* p), (p)) { return mi_usable_size(p); }
INTERPOSE_C(void*, valloc, (size_t n), (n)) { return mi_valloc(n); }
INTERPOSE_C(void*, pvalloc, (size_t n), (n)) { return mi_pvalloc(n); }
INTERPOSE_C(void*, reallocarray, (void* p, size_t s, size_t n), (p, s, n)) { return mi_reallocarray(p, s, n); }
INTERPOSE_C(void*, memalign, (size_t a, size_t n), (a, n)) { return mi_memalign(a, n); }
INTERPOSE_C(void*, aligned_alloc, (size_t a, size_t n), (a, n)) { return mi_aligned_alloc(a, n); }
INTERPOSE_C(int, posix_memalign, (void** p, size_t a, size_t n), (p, a, n)) { return mi_posix_memalign(p, a, n); }
#else
#include <interpose.hh>
INTERPOSE(malloc)(size_t n) { return mi_malloc(n); }
INTERPOSE(calloc)(size_t n, size_t c) { return mi_calloc(n, c); }
INTERPOSE(realloc)(void* p, size_t n) { return mi_realloc(p, n); }
INTERPOSE(free)(void* p) { return mi_free(p); }
INTERPOSE(strdup)(const char* s) { return mi_strdup(s); }
INTERPOSE(strndup)(const char* s, size_t n) { return mi_strndup(s, n); }
INTERPOSE(realpath)(const char* f, char* n) { return mi_realpath(f, n); }
INTERPOSE(malloc_usable_size)(void* p) { return mi_usable_size(p); }
INTERPOSE(valloc)(size_t n) { return mi_valloc(n); }
INTERPOSE(pvalloc)(size_t n) { return mi_pvalloc(n); }
INTERPOSE(reallocarray)(void* p, size_t s, size_t n) { return mi_reallocarray(p, s, n); }
INTERPOSE(memalign)(size_t a, size_t n) { return mi_memalign(a, n); }
INTERPOSE(aligned_alloc)(size_t a, size_t n) { return mi_aligned_alloc(a, n); }
INTERPOSE(posix_memalign)(void** p, size_t a, size_t n) { return mi_posix_memalign(p, a, n); }
#endif // defined(__cplusplus)
#endif // MIMALLOC_OVERRIDE_H
This is just a Proof of Concept, and many things have not been taken into account, but I hope it helps
but I have to set the
--allow-multiple-definitionflag because multiple implementations of the same__interpose_are created
Uh, but that would suggest to me that you should interpose a specific function once.
The interpose readme also says "#include <interpose.hh> in the source file where you will implement the replacement functions", also suggesting to me this should go into a source and compiled into one object, not into a header...
use (std::realloc)(Ptr, size)
btw, same on windows for std::min/max, if macro NOMINMAX not defined.
