Add macros to set calling conventions for callbacks.

[seanmiddleditch]

Required to successfully link when compiling with non-default calling conventions.

e.g., if the hosting app is compiled with /Gv to use the __vectorcall calling convention by default, which is sometimes a huge performance improvement, they will not be able to call imgui functions. Compiling imgui itself with /Gv does not work since system callbacks also need to be marked.

This change makes a single universal IMGUI_CALLBACK macro for marking up all callbacks, typically to __cdecl. If there is a desire to allow imgui callbacks to have custom calling conventions (if there is ever a high-volume callback, for instance) then there may need to be separate macros for CRT callbacks and imgui callbacks.

This change also modified an stb header that uses qsort and hence needs to set calling convention. That change could be pushed back to stb if there is a desire to avoid patches over stb.

[seanmiddleditch]

Do you mean none of the imgui functions can be called?

In the case that the host app is compiled with /Gv and imgui is in a static library compiled with a different default ABI, then no, you not be able to invoke any imgui function (and will get a linker error because the ABI affects mangling).

If you compile imgui with /Gv then the standard imgui functions are invokable but callbacks become problematic due to a mixture of system-provided functions and user-provided functions used as callbacks (e.g., the IME and clipboard callbacks, among others, iirc). The system-provided functions on Windows will always use __cdecl but user-provided ones will only do so if they explicitly set it as such. imgui could use wrappers around all system-provided functions to avoid ABI issues, but that adds some library bloat and is cumbersome.

Since tossing __cdecl into user code makes said code unportable, the common approach here is to use a macro on all callback functions that expands to __cdecl on Windows and expands to nothing on other platforms.

[ocornut]

Hello @seanmiddleditch, and my apologies for keeping that PR dangling for so long.

I recently merged in #1611 which seems to tackle (partly) the same issue but only changed the 3 functions (there was 2 at the time of your PR) that are called by e.g. libc qsort.

I am a little confused however about the extra changes affecting e.g. GetClipboardTextFn, SetClipboardTextFn, ImeSetInputScreenPosFn. Just to clarify - if I understand correctly, the extra changes are necessary only to allow compiling your app code (which would include imgui.h and potentially assign to those function pointers) and imgui.cpp with different calling conventions? Is that correct? If the full app + imgui are compiled with the same calling convention, e.g. /Gv it doesn't seem to be a problem? (since we fixed the function called by qsort in #1611).

[seanmiddleditch]

Hi, yeah it would affect things if calling connections differ. Not my use case but still a potentially valid one to consider.

[ocornut]

Hello Sean, Any reason for closing this?

[seanmiddleditch]

@ocornut - ah, don't think I did that intentionally, and never got through my github notifications backlog until now to see this question :( -- cool to reopen?

[ocornut]

Sure!

[seanmiddleditch]

Cool, thanks. It's been a few years, but... are you on board with the general gist of fixes? I can get this updated to the latest master.

Since you're on Github Actions now I can also add a test specifically for the case I ran into with using /Gv as a global compilation option, as well as a test with mixing/matching the calling conventions between imgui and test code.