Static override on Windows

[aganea]

Hello!

Would it be possible to provide something like a static-override.c along with mimalloc? Other allocators have such implementation which allows to easily integrate mimalloc into a codebase & compile with the codebase's compiler and options. See https://github.com/mjansson/rpmalloc/blob/develop/rpmalloc/malloc.c Or https://github.com/microsoft/snmalloc/blob/master/src/override/malloc.cc

By applying the patch below, mimalloc can be linked as-a-lib along with said codebase. However when using optimization options or LTO/LTCG compilation, it'd be nice if the mimalloc source would be compiled into the codebase (considering /MT is being used, not /MD)

Many thanks! Alex.

diff --git a/ide/vs2019/mimalloc.vcxproj b/ide/vs2019/mimalloc.vcxproj
index e18db0c..cdf2a0e 100644
--- a/ide/vs2019/mimalloc.vcxproj
+++ b/ide/vs2019/mimalloc.vcxproj
@@ -171,7 +171,7 @@
       <FunctionLevelLinking>true</FunctionLevelLinking>
       <ConformanceMode>true</ConformanceMode>
       <AdditionalIncludeDirectories>../../include</AdditionalIncludeDirectories>
-      <PreprocessorDefinitions>%(PreprocessorDefinitions);NDEBUG</PreprocessorDefinitions>
+      <PreprocessorDefinitions>%(PreprocessorDefinitions);NDEBUG;MI_MALLOC_OVERRIDE</PreprocessorDefinitions>
       <AssemblerOutput>AssemblyAndSourceCode</AssemblerOutput>
       <AssemblerListingLocation>$(IntDir)</AssemblerListingLocation>
       <WholeProgramOptimization>false</WholeProgramOptimization>
@@ -180,6 +180,7 @@
       <CompileAs>CompileAsCpp</CompileAs>
       <IntrinsicFunctions>true</IntrinsicFunctions>
       <LanguageStandard>Default</LanguageStandard>
+      <RuntimeLibrary>MultiThreaded</RuntimeLibrary>
     </ClCompile>
     <Link>
       <EnableCOMDATFolding>true</EnableCOMDATFolding>
diff --git a/src/alloc-override.c b/src/alloc-override.c
index a09153c..e9344ed 100644
--- a/src/alloc-override.c
+++ b/src/alloc-override.c
@@ -10,10 +10,10 @@ terms of the MIT license. A copy of the license can be found in the file
 #endif
 
 #if defined(MI_MALLOC_OVERRIDE) && defined(_WIN32) && !(defined(MI_SHARED_LIB) && defined(_DLL))
-#error "It is only possible to override "malloc" on Windows when building as a DLL (and linking the C runtime as a DLL)"
+//#error "It is only possible to override "malloc" on Windows when building as a DLL (and linking the C runtime as a DLL)"
 #endif
 
-#if defined(MI_MALLOC_OVERRIDE) && !(defined(_WIN32)) // || (defined(__MACH__) && !defined(MI_INTERPOSE)))
+#if defined(MI_MALLOC_OVERRIDE) //&& !(defined(_WIN32)) // || (defined(__MACH__) && !defined(MI_INTERPOSE)))
 
 // ------------------------------------------------------
 // Override system malloc

[daanx]

Hi, apologies for the late reply. I think this is about overriding malloc on Windows? In that case, we unfortunately cannot override the standard malloc by just linking with a library or object file (as in the readme). (it works on Linux though using static.c)

Perhaps I do not understand the question well?

[aganea]

@daanx With the above patch, it does work if mimalloc is compiled along (embedded into) the application, and iff it's compiled with /MT (static CRT). Competing allocators (snmalloc and rpmalloc) support this use-case (statically override malloc). Do you think this could be added to mimalloc? Please see the last comments and that patch here for more info: https://reviews.llvm.org/D71786

[rvjr]

Hi @daanx,

I'm actually facing a related issue at the moment: I was trying to replace the allocator in a rather large DLL project with multiple library dependencies. CRT and dependent libs are all statically built into the DLL in order to have zero dependencies from the final DLL. As all source is under my control I though it would be fine to use forced includes (in MSVC command line: /FI"mimalloc-override.h") on every subproject to inject the new allocator into every source file.

This method is mentioned in the readme as a possible way of overriding statically on Windows. But it gets me into trouble with CRT headers. I cleaned up the build output a bit, and the important parts look like this:

1>------ Build started: Project: mimalloc-override-test, Configuration: Debug x64 ------
1>Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30038.1 for x64
1>cl /c /I..\..\include /ZI /W3 /WX- /diagnostics:column /sdl /Od /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /permissive- /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"c:\data\Projects\SDK-master\common\external\mimalloc\ide\vs2019\..\..\out\msvc-x64\mimalloc-override-test\Debug\\" /Fd"c:\data\Projects\SDK-master\common\external\mimalloc\ide\vs2019\..\..\out\msvc-x64\mimalloc-override-test\Debug\vc142.pdb" /external:env:EXTERNAL_INCLUDE /external:W3 /Gd /TP /FI"mimalloc-override.h" /showIncludes /FC /errorReport:prompt "..\..\test\main-override.cpp"
1>main-override.cpp
1>Note: including file: c:\data\Projects\SDK-master\common\external\mimalloc\include\mimalloc-override.h
1>Note: including file:  c:\data\Projects\SDK-master\common\external\mimalloc\include\mimalloc.h
1>Note: including file: C:\Program Files (x86)\Windows Kits\10\Include\10.0.17763.0\ucrt\string.h
1>Note: including file:  C:\Program Files (x86)\Windows Kits\10\Include\10.0.17763.0\ucrt\corecrt_wstring.h
1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.17763.0\ucrt\corecrt_wstring.h(79,1): error C2059: syntax error: '('

The injected macros of mimalloc seem to interfere with the function declarations they should override, like in this case _wcsdup in corecrt_wstring.h:

_Check_return_ _ACRTIMP _CRTALLOCATOR wchar_t* __cdecl _wcsdup( _In_z_ wchar_t const* _String );

The issue is easily reproduced by adding /FI"mimalloc-override.h" in the mimalloc-override-test project (I used MSVC2019).

Am I doing something completely wrong? Do you have a suggestion how to fix this?

Best regards, Rainer

[rvjr]

@daanx I maybe found a suitable solution for my case:

• it seems to be too complicated to inject mimalloc everywhere using the /FI switch; it just doesn't compile and errors seem to be hard to fix
• I'm not in control of the executable of my shared library project (a windows DLL), so unfortunately minject (which works well) is not a solution

Instead I now resorted to overriding new and delete only.

• It is sufficient for me to have allocations of C++ objects and STL containers in mimalloc, because that's where the core, high-performance code lives
• Some other static libraries in my DLL project, or the code in the loading executable itself, will still use malloc/free from the default CRT allocator, which is ok for me. I don't use malloc/free in my own code though.
• I managed to get everything working quickly by simply linking the library against mimalloc-static and include mimalloc-new-delete.h in one of my source files. new and delete are overridden in the entire shared library, and everything looks fine now.

Do you see any issue with this procedure?

Best regards, Rainer

[rc-power]

try to use static override：

"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.43.34808\\bin\\HostX64\\x64\\link.exe" -nologo -dynamicbase -nxcompat -machine:x64 -libpath:build\windows\x64\debug -libpath:"C:\Program Files (x86)\Windows Kits\10\Lib\10.0.22621.0\um\x64" -libpath:"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.43.34808\lib\onecore\x64" -libpath:"C:\Program Files (x86)\Windows Kits\10\Lib\10.0.22621.0\ucrt\x64" -debug -pdb:build\windows\x64\debug\test.pdb -nodefaultlib:msvcrt.lib mimalloc.lib /INCLUDE:_dupenv_s tracy.lib psapi.lib shell32.lib user32.lib advapi32.lib bcrypt.lib ws2_32.lib dbghelp.lib -out:build\windows\x64\debug\test.exe build\.objs\test\windows\x64\debug\main_test.cpp.obj

but get this:

libucrtd.lib(getenv.obj) : error LNK2005: _dupenv_s 已经在 mimalloc.lib(static.c.obj) 中定义
libucrtd.lib(getenv.obj) : error LNK2005: _wdupenv_s 已经在 mimalloc.lib(static.c.obj) 中定义
libucrtd.lib(expand.obj) : error LNK2005: _expand 已经在 mimalloc.lib(static.c.obj) 中定义
`build\windows\x64\debug\test.exe` : fatal error LNK1169: 找到一个或多个多重定义的符号

looks like these symbols unavoidable