dlang
Implement some of MSVC's intrinsics, for ImportC.
This PR implements all but three of the MSVC intrinsics listed here: https://web.archive.org/web/20240412171516/https://learn.microsoft.com/en-ie/cpp/intrinsics/alphabetical-listing-of-intrinsic-functions?view=msvc-170, and a handful of undocumented intrinsics.
The three unimplemented intrinsics are _AddressOfReturnAddress, __getcallerseflags, and _ReturnAddress, because they need compiler support for their implementation.
The implemented intrinsics are listed in this expando.
__assume__umulh__mulh_umul128_mul128__emul__emulu_div128_udiv128_div64_udiv64__cpuid__cpuidex_cvt_ftoi_fast_cvt_ftoll_fast_cvt_ftoui_fast_cvt_ftoull_fast_cvt_dtoi_fast_cvt_dtoll_fast_cvt_dtoui_fast_cvt_dtoull_fast_cvt_ftoi_sat_cvt_ftoll_sat_cvt_ftoui_sat_cvt_ftoull_sat_cvt_dtoi_sat_cvt_dtoll_sat_cvt_dtoui_sat_cvt_dtoull_sat_cvt_ftoi_sent_cvt_ftoll_sent_cvt_ftoui_sent_cvt_ftoull_sent_cvt_dtoi_sent_cvt_dtoui_sent_cvt_dtoll_sent_cvt_dtoull_sent__readgsbyte__readgsword__readgsdword__readgsqword__writegsbyte__writegsword__writegsdword__writegsqword__addgsbyte__addgsword__addgsdword__addgsqword__incgsbyte__incgsword__incgsdword__incgsqword__readfsbyte__readfsword__readfsdword__readfsqword__writefsbyte__writefsword__writefsdword__writefsqword__addfsbyte__addfsword__addfsdword__incfsbyte__incfsword__incfsdword__debugbreak__fastfail__faststorefence_disable_enable_interlockedadd [Undocumented]_interlockedadd64 [Undocumented]_InterlockedAdd_InterlockedAdd_acq_InterlockedAdd_rel_InterlockedAdd_nf_InterlockedAdd64_InterlockedAdd64_acq_InterlockedAdd64_rel_InterlockedAdd64_nf_InterlockedAddLargeStatistic_InterlockedAnd_InterlockedAnd8_InterlockedAnd16_interlockedand64 [Undocumented]_InterlockedAnd_acq_InterlockedAnd_rel_InterlockedAnd_nf_InterlockedAnd8_acq_InterlockedAnd8_rel_InterlockedAnd8_nf_InterlockedAnd16_acq_InterlockedAnd16_rel_InterlockedAnd16_nf_InterlockedAnd64_acq_InterlockedAnd64_rel_InterlockedAnd64_nf_InterlockedAnd64_InterlockedAnd_np_InterlockedAnd8_np_InterlockedAnd16_np_InterlockedAnd64_np_InterlockedAnd64_HLEAcquire_InterlockedAnd64_HLERelease_InterlockedAnd_HLEAcquire_InterlockedAnd_HLERelease_interlockedbittestandreset_interlockedbittestandreset64_interlockedbittestandreset_HLEAcquire_interlockedbittestandreset_HLERelease_interlockedbittestandreset64_HLEAcquire_interlockedbittestandreset64_HLERelease_interlockedbittestandreset_acq_interlockedbittestandreset_rel_interlockedbittestandreset_nf_interlockedbittestandreset64_acq_interlockedbittestandreset64_rel_interlockedbittestandreset64_nf_interlockedbittestandset_interlockedbittestandset64_interlockedbittestandset_HLEAcquire_interlockedbittestandset_HLERelease_interlockedbittestandset64_HLEAcquire_interlockedbittestandset64_HLERelease_interlockedbittestandset_acq_interlockedbittestandset_rel_interlockedbittestandset_nf_interlockedbittestandset64_acq_interlockedbittestandset64_rel_interlockedbittestandset64_nf_InterlockedCompareExchange_InterlockedCompareExchange8_InterlockedCompareExchange16_InterlockedCompareExchange64_InterlockedCompareExchange_HLEAcquire_InterlockedCompareExchange_HLERelease_InterlockedCompareExchange64_HLEAcquire_InterlockedCompareExchange64_HLERelease_InterlockedCompareExchange_np_InterlockedCompareExchange16_np_InterlockedCompareExchange64_np_InterlockedCompareExchange_acq_InterlockedCompareExchange_rel_InterlockedCompareExchange_nf_InterlockedCompareExchange8_acq_InterlockedCompareExchange8_rel_InterlockedCompareExchange8_nf_InterlockedCompareExchange16_acq_InterlockedCompareExchange16_rel_InterlockedCompareExchange16_nf_InterlockedCompareExchange64_acq_InterlockedCompareExchange64_rel_InterlockedCompareExchange64_nf_InterlockedCompareExchange128_InterlockedCompareExchange128_np_InterlockedCompareExchange128_acq_InterlockedCompareExchange128_rel_InterlockedCompareExchange128_nf_InterlockedCompareExchangePointer_InterlockedCompareExchangePointer_HLEAcquire_InterlockedCompareExchangePointer_HLERelease_InterlockedCompareExchangePointer_np_InterlockedCompareExchangePointer_acq_InterlockedCompareExchangePointer_rel_InterlockedCompareExchangePointer_nf_InterlockedDecrement_InterlockedDecrement16_interlockeddecrement64_InterlockedDecrement64_InterlockedDecrement_acq_InterlockedDecrement_rel_InterlockedDecrement_nf_InterlockedDecrement16_acq_InterlockedDecrement16_rel_InterlockedDecrement16_nf_InterlockedDecrement64_acq_InterlockedDecrement64_rel_InterlockedDecrement64_nf_InterlockedExchange_InterlockedExchange8_InterlockedExchange16_interlockedexchange64_InterlockedExchange64_InterlockedExchange_HLEAcquire_InterlockedExchange_HLERelease_InterlockedExchange64_HLEAcquire_InterlockedExchange64_HLERelease_InterlockedExchange_acq_InterlockedExchange_rel_InterlockedExchange_nf_InterlockedExchange8_acq_InterlockedExchange8_rel_InterlockedExchange8_nf_InterlockedExchange16_acq_InterlockedExchange16_rel_InterlockedExchange16_nf_InterlockedExchange64_acq_InterlockedExchange64_rel_InterlockedExchange64_nf_InterlockedExchangeAdd_InterlockedExchangeAdd8_InterlockedExchangeAdd16_interlockedexchangeadd64 [Undocumented]_InterlockedExchangeAdd64_InterlockedExchangeAdd_HLEAcquire_InterlockedExchangeAdd_HLERelease_InterlockedExchangeAdd64_HLEAcquire_InterlockedExchangeAdd64_HLERelease_InterlockedExchangeAdd_acq_InterlockedExchangeAdd_rel_InterlockedExchangeAdd_nf_InterlockedExchangeAdd8_acq_InterlockedExchangeAdd8_rel_InterlockedExchangeAdd8_nf_InterlockedExchangeAdd16_acq_InterlockedExchangeAdd16_rel_InterlockedExchangeAdd16_nf_InterlockedExchangeAdd64_acq_InterlockedExchangeAdd64_rel_InterlockedExchangeAdd64_nf_InterlockedExchangePointer_InterlockedExchangePointer_HLEAcquire_InterlockedExchangePointer_HLERelease_InterlockedExchangePointer_acq_InterlockedExchangePointer_rel_InterlockedExchangePointer_nf_InterlockedIncrement_InterlockedIncrement16_interlockedincrement64 [Undocumented]_InterlockedIncrement64_InterlockedIncrement_acq_InterlockedIncrement_rel_InterlockedIncrement_nf_InterlockedIncrement16_acq_InterlockedIncrement16_rel_InterlockedIncrement16_nf_InterlockedIncrement64_acq_InterlockedIncrement64_rel_InterlockedIncrement64_nf_InterlockedOr_InterlockedOr8_InterlockedOr16_interlockedor64 [Undocumented]_InterlockedOr_acq_InterlockedOr_rel_InterlockedOr_nf_InterlockedOr8_acq_InterlockedOr8_rel_InterlockedOr8_nf_InterlockedOr16_acq_InterlockedOr16_rel_InterlockedOr16_nf_InterlockedOr64_acq_InterlockedOr64_rel_InterlockedOr64_nf_InterlockedOr64_InterlockedOr_np_InterlockedOr8_np_InterlockedOr16_np_InterlockedOr64_np_InterlockedOr64_HLEAcquire_InterlockedOr64_HLERelease_InterlockedOr_HLEAcquire_InterlockedOr_HLERelease_InterlockedXor_InterlockedXor8_InterlockedXor16_interlockedxor64 [Undocumented]_InterlockedXor_acq_InterlockedXor_rel_InterlockedXor_nf_InterlockedXor8_acq_InterlockedXor8_rel_InterlockedXor8_nf_InterlockedXor16_acq_InterlockedXor16_rel_InterlockedXor16_nf_InterlockedXor64_acq_InterlockedXor64_rel_InterlockedXor64_nf_InterlockedXor64_InterlockedXor_np_InterlockedXor8_np_InterlockedXor16_np_InterlockedXor64_np_InterlockedXor64_HLEAcquire_InterlockedXor64_HLERelease_InterlockedXor_HLEAcquire_InterlockedXor_HLERelease__inbyte__inword__indword__outbyte__outword__outdword__inbytestring__inwordstring__indwordstring__outbytestring__outwordstring__outdwordstring__int2c__invlpg__lidt__ll_lshift__ll_rshift__ull_rshift__lzcnt16__lzcnt__lzcnt64_mm_cvtsi64x_ss_mm_cvtss_si64x_mm_cvttss_si64x_mm_extract_si64_mm_extracti_si64_mm_insert_si64_mm_inserti_si64_mm_stream_sd_mm_stream_ss_mm_stream_si64x__movsb__movsw__movsd__movsq__noop__nop__popcnt16__popcnt__popcnt64__rdtsc__rdtscp__readcr0__readcr2__readcr3__readcr4__readcr8__readdr__readeflags__readmsr__readpmc__segmentlimit__shiftleft128__shiftright128__sidt__stosb__stosw__stosd__stosq__svm_clgi__svm_invlpga__svm_skinit__svm_stgi__svm_vmload__svm_vmrun__svm_vmsave__ud2__vmx_off__vmx_on__vmx_vmclear__vmx_vmlaunch__vmx_vmptrld__vmx_vmptrst__vmx_vmread__vmx_vmresume__vmx_vmwrite__wbinvd__writecr0__writecr2__writecr3__writecr4__writecr8__writedr__writeeflags__writemsr_ReadBarrier_WriteBarrier_ReadWriteBarrier_BitScanForward_BitScanReverse_BitScanForward64_BitScanReverse64_bittest_bittestandcomplement_bittestandreset_bittestandset_bittest64_bittestandcomplement64_bittestandreset64_bittestandset64_byteswap_uint64_byteswap_ulong_byteswap_ushort_lrotr_lrotl_rotr_rotl_rotr64_rotl64_rotr16_rotl16_rotr8_rotl8
These implementations aim to be as compatible with the MSVC intrinsics as is possible—adhering to Hyrum's Law.
Separate implementations are provided for DMD, LDC, and GDC, and for x86, x86-64, AArch64, and ARM.
Almost all the intrinsics are implemented in D, except for __assume which is a C macro.
Every intrinsic, where possible, has a CTFE-compatible code-path.
It all compiles with or without DIP1000 being enabled.
Care has been taken to ensure that none of the implementations rely on DRuntime, so that these work in BetterC.
Regarding the _cvt_ family of functions: by default MSVC will generate code that uses SSE2 instructions, even for 32-bit targets, which means that for 32-bit targets the _cvt_ functions will use SSE2.
This is contrary to DMD's usual behaviour of using x87 for 32-bit Windows.
For their reimplementations, I've used SSE2 anyway for 32-bit targets for DMD, as doing otherwise would constitute a change in behaviour, as x87 FP-exceptions are different from SIMD FP-exceptions (and, I think the SSE2 and x87 versions return different results).
The oldest of the _cvt_ functions was introduced in the May of 2021, so I think it's almost certain that any code using them will be targeting at-least SSE2 anyway.
Additionally, I've written a program that tests that the _cvt_ implementations return identical results to the MSVC implementations for all float values, and for ~402,653,184 double values (except for _cvt_ftoi_fast and _cvt_ftoi_sent on 32-bit targets, as they cause an internal compiler error in MSVC); it also tests that ctfeX86RoundLongToFloat and ctfeX86RoundFloatToLong produce the same results as the hardware.
It relies on Phobos and MSVC, so I don't really know what to do with it other than link to it here: https://github.com/just-harry/float-fuzzing-for-msvc-intrinsics
A few of the intrinsics can be used only in kernel-mode, so unittests have been omitted for them, as I don't think we have any infrastructure in place for testing in kernel-mode.
Some of the intrinsics terminate the program, so their unittests have been wrapped in a version (none), others rely on specific compiler optimisations, so they too have been wrapped in a version (none).
I've split the intrinsics up into a few dozen commits to try and alleviate the whole 13,000-lines-of-codes-all-at-once thing.
(I staged them after-the-fact, so a few braces may be out-of-place in the intermediate commits.)
The intrinsics have been placed in their own files, separate from the existing ImportC builtin files, to try and avoid crowding the builtin files.
Currently, the ImportC builtins are imported conditionally, based on some heuristics. One of those heuristics is if any identifier beginning with two underscores is used – I've changed that one to instead trigger on a single leading underscore as many of MSVC's intrinsics begin with only one underscore.
One notable header that can be successfully included by ImportC, with these intrinsics implemented, is windows.h.
P.S. If this is outside the purview of ImportC: that's fine, I'll just publish this as a library instead.
I've flipped this back to a draft PR as currently the functions in the __builtins_msvc module can be used only when the -i switch is supplied on the command-line.
I think it might be that semantic3 isn't getting run on that module's functions when -i isn't supplied—but I'm unsure.
This is still a draft as I'm unable to get the compiler to actually generate code for the __builtins_msvc module without its path being supplied on the command-line.
Presently, neither the __builtins module nor the __builtins_msvc module make their way into the array of modules passed into generateCodeAndWrite, which is fine for __builtins, but not for __builtins_msvc because of how the standard Windows headers use the builtins, and because DMD doesn't inline some of the asm-using functions.
This is effectively how winnt.h declares UnsignedMultiply128:
unsigned long long UnsignedMultiply128 (unsigned long long Multiplier, unsigned long long Multiplicand, unsigned long long *HighProduct);
#define UnsignedMultiply128 _umul128
unsigned long long UnsignedMultiply128 (unsigned long long Multiplier, unsigned long long Multiplicand, unsigned long long *HighProduct);
which will expand to unsigned long long _umul128 (unsigned long long Multiplier, unsigned long long Multiplicand, unsigned long long *HighProduct);, causing the linker to expect code for _umul128.
But even if the Windows headers didn't do that, we still need code to be generated for _umul128, because DMD won't inline _umul128's call to multiplyWithDoubleWidthProduct and the linker will expect code for it.
(_umul128 isn't the only function affected.)
Right now I'm stuck, because I've yet to figure out a good way to get the __builtins_msvc module into the modules array passed into generateCodeAndWrite after it's conditionally imported if any MSVC intrinsics are detected.
The best I've come up with thus far is very hacky, and would require the same changes to be made in LDC and GDC (unless I'm mistaken), which is: in Module.parseModule, if the module identifier is __builtins_msvc, set the global includeImports to true, and if it was previously false set includeByDefault to false, then add __builtins_msvc to the compiledImports array.
In short, my question is: is there an existing way to make it so the __builtins_msvc module winds up getting passed to generateCodeAndWrite after it gets imported by CParser.parseModule?
And, if not, is there an acceptably non-hacky way to thread it through the compiler without requiring changes to LDC and GDC?
Any suggestions would be appreciated.
I suggest having the user do the #include line as the first line in his C code. Or have the user pass it in with the dmd switch that passes switches to the C preprocessor. Will that work?
@WalterBright,
I suggest having the user do the #include line as the first line in his C code. Or have the user pass it in with the dmd switch that passes switches to the C preprocessor. Will that work?
If that works for you, it works for me. I don't see an issue with:
#include <importc_msvc_builtins.h>
#include <windows.h>
I've added some code to supply <path>/importc.h/../include to the preprocessor as an include path, so that the user doesn't have to hunt down the path themselves. (https://github.com/dlang/dmd/pull/16372/commits/de631404562ee6ba353d335efda8c5905191b7b3)
Though, the issue of these builtins only working when the -i switch is used still remains.
Is there an existing way to force codegen for an imported module, when the -i switch isn't used?
Okay; a separate PR adding a mechanism that this PR can use to force codegen for the __builtins_msvc module: https://github.com/dlang/dmd/pull/16443
Via something like this in cparse.d:
if (token.value == TOK._import) // import declaration extension
{
auto a = parseImport();
if (a && a.length)
/*+*/ {
/*+*/ auto imp = (*a)[0].isImport();
/*+*/ imp.forceCodegen = imp.id == Id.builtins_msvc;
symbols.append(a);
/*+*/ }
return;
}
Though, the issue of these builtins only working when the
-iswitch is used still remains.Is there an existing way to force codegen for an imported module, when the
-iswitch isn't used?
Did you try converting these functions to templates?
Though, the issue of these builtins only working when the
-iswitch is used still remains. Is there an existing way to force codegen for an imported module, when the-iswitch isn't used?Did you try converting these functions to templates?
I did—but no dice, unfortunately, even with all the affected functions being templates, the simplest possible usage still fails to link:
// windows_h_c.c:
#include <importc_msvc_builtins.h>
#include <windows.h>
// windows_h.d:
pragma(lib, "MinCore");
import windows_h_c;
windows_h.obj : error LNK2019: unresolved external symbol _InterlockedExchangeAdd referenced in function _InlineInterlockedAdd
windows_h.obj : error LNK2019: unresolved external symbol _InterlockedExchangeAdd64 referenced in function _InlineInterlockedAdd64
windows_h.obj : error LNK2019: unresolved external symbol _mul128 referenced in function MultiplyExtract128
windows_h.obj : error LNK2019: unresolved external symbol __shiftright128 referenced in function MultiplyExtract128
windows_h.obj : error LNK2019: unresolved external symbol _umul128 referenced in function UnsignedMultiplyExtract128
windows_h.obj : error LNK2019: unresolved external symbol _ReadWriteBarrier referenced in function BarrierAfterRead
windows_h.obj : error LNK2019: unresolved external symbol __stosb referenced in function RtlSecureZeroMemory
windows_h.obj : error LNK2019: unresolved external symbol __readgsqword referenced in function NtCurrentTeb
windows_h.exe : fatal error LNK1120: 8 unresolved externals
Just have the user add the #include line.
Perhaps I'm misunderstanding you, but is that not what the current implementation is doing? In the example here: https://github.com/dlang/dmd/pull/16372#issuecomment-2094644837
There, the user #includes <importc_msvc_builtins.h> before <windows.h>, which works perfectly well.
importc_msvc_builtins.h then does an __import __builtins_msvc;
The import of __builtins_msvc succeeds because druntime/import is in the compiler's import path.
But because the path to __builtins_msvc.d wasn't explicitly provided on the command-line, codegen doesn't take place for the __builtins_msvc module.
Which ultimately causes the linker to fail because the way the standard Windows headers uses some of these intrinsics causes the linker to expect a definition for them.
windows_h.obj : error LNK2019: unresolved external symbol _InterlockedExchangeAdd referenced in function _InlineInterlockedAdd
I can try to synthesize this later (Linux). But it would be useful to know how these symbols are being referenced in the windows headers.
My immediate suspicion is either the address is taken, or windows.h adds its own extern declaration outside of importc msvc builtins.
windows_h.obj : error LNK2019: unresolved external symbol _InterlockedExchangeAdd referenced in function _InlineInterlockedAdd
I can try to synthesize this later (Linux). But it would be useful to know how these symbols are being referenced in the windows headers.
My immediate suspicion is either the address is taken, or windows.h adds its own extern declaration outside of importc msvc builtins.
I think it's the latter.
For example, for __shiftright128 in winnt.h, there's
#define ShiftRight128 __shiftright128
DWORD64
ShiftRight128 (
_In_ DWORD64 LowPart,
_In_ DWORD64 HighPart,
_In_ BYTE Shift
);
#pragma intrinsic(__shiftright128)
Similarly, for _mul128 in winnt.h:
#define Multiply128 _mul128
LONG64
Multiply128 (
_In_ LONG64 Multiplier,
_In_ LONG64 Multiplicand,
_Out_ LONG64 *HighProduct
);
#pragma intrinsic(_mul128)
The same pattern is used for the rest of affected functions, in winnt.h; with the minor difference of no #define being used for _ReadWriteBarrier, __stosb, or __readgsqword.
But because the path to __builtins_msvc.d wasn't explicitly provided on the command-line, codegen doesn't take place for the __builtins_msvc module.
For now I suggest just asking the user to put it on the command line.
But because the path to __builtins_msvc.d wasn't explicitly provided on the command-line, codegen doesn't take place for the __builtins_msvc module.
For now I suggest just asking the user to put it on the command line.
If it comes down to it, yeah okay. But I don't think it makes for a great user-experience, and the point of ImportC is to reduce friction, so I'd like to try one last thing.
With an addition to ImportC, we can make it so these intrinsics can be inlined, and so that they can be templates (removing the need for unconditional codegen for the __builtins_msvc module)—improving runtime performance, reducing compile-times, and making them easier to use: https://github.com/dlang/dmd/pull/16464