ldc Calling convention surprises

Check this out: https://godbolt.org/z/1cYPbMdz5

I'm on x86. I started by trying to specify fastcall on a function, but then I got something that looked weird, so I pulled out compiler explorer, and everything looks weird except extern(C).

You'll notice in that experiment:

extern(C) looks correct, 2 args pushed on the stack
extern(D) looks odd, I thought it was meant to match C (I often pass D function pointers to C code!), but it seems to push just one arg on the stack, and pass the first arg in EAX for some reason?
When I add @callingConvention("fastcall"), I definitely don't get fastcall (which would be 2 int args passed in ECX, EDX), it just looks the same as extern(D), EXCEPT, for reasons unknown, it decides to put the arg in ECX instead of EAX... that's weird!

So, does anyone know what's going on here?

What I really want is for @callingConvention("fastcall") to work, which is an ABI where the first 2 integer args are passed in ECX, EDX.

Mar 17 '25 14:03 TurkeyMan

@callingConvention is highly experimental, I suggest not using it. Do you really still have to deal with 32-bit x86 code?

Mar 17 '25 14:03 kinke

I mean, it's a supported target, but it's low priority.

That aside though, what's with extern(D)? That looks broken too... I never saw anywhere that D has a custom ABI?

Mar 17 '25 14:03 TurkeyMan

Wrt. extern(D), that uses Walter's custom ABI for Win32, trying to use EAX for passing an argument: https://dlang.org/spec/abi.html#function_calling_conventions. The spec says it's for Win32 only, but it's actually used for all 32-bit x86 targets. DMD additionally reverses the args for extern(D), which LDC and GDC don't.

Mar 17 '25 14:03 kinke

The extern (C) and extern (D) calling convention matches the C calling convention used by the supported C compiler on the host system.
Except that the extern (D) calling convention for Windows x86 is described here.

Ah damn, I actually read that paragraph, but I managed to only read the first sentence and didn't see the "Except..." bit. So, is the spec wrong? Why does it say windows x86, but it applies everything?

And if as you say DMD pushes the args in reverse, does that mean DMD and LDC/GDC binaries are incompatible? O_o

Mar 17 '25 14:03 TurkeyMan

But yeah, fastcall is a common ABI, I kinda need it. Otherwise I gotta write shim's and stuff, which rather defeats the purpose of fastcall :P

Mar 17 '25 14:03 TurkeyMan

@callingConvention is so experimental that you don't even get an error when specifying an invalid one :D - try extern(C) @callingConvention("fastcc"), that works. See https://llvm.org/docs/LangRef.html#calling-conventions for available stuff, but again, I highly discourage using that UDA.

Mar 17 '25 14:03 kinke

Why discourage it? Why not make it work good?

Mar 17 '25 14:03 TurkeyMan

IMO not worth the effort, uninteresting 32-bit x86 legacy stuff.

Mar 17 '25 14:03 kinke

you don't even get an error when specifying an invalid one :D

This doesn't seem insurmountable :P

Mar 17 '25 14:03 TurkeyMan

IMO not worth the effort, uninteresting 32-bit x86 legacy stuff.

Well, it seems to work; fastcc did what I expect... so, why shouldn't I use it exactly?

Mar 17 '25 14:03 TurkeyMan

That said though, the weird case where it was still extern(D) but the register randomly swapped over... that's really weird!

Mar 17 '25 14:03 TurkeyMan

so, why shouldn't I use it exactly?

Because it's a blunt instrument to override the CC as LLVM attribute, totally ignoring what we have to do laboriously in the gen/abi/ ABI implementations to implement all kinds of ABI quirks, and getting LLVM to do what we want. E.g., for x86 and extern(D): https://github.com/ldc-developers/ldc/blob/61c11b7dafa24d9c67011d03a0b63d3287791c10/gen/abi/x86.cpp#L175-L221

Mar 17 '25 15:03 kinke

I would have imagined that you didn't have to write or maintain fastcc; I would have imagined it's just hanging out in an enum somewhere in LLVM somewhere because it's used by Clang...?

I agree that D having a custom ABI is ridiculous, there's already more than enough!

Mar 17 '25 15:03 TurkeyMan

It's not just this custom x86 ABI for D, but e.g. also the regular SysV ABI for non-Windows x86_64 - it's not like LLVM does everything for us, as is the case for GDC apparently. Getting the ABI incl. calling conventions right is a huge PITA for LDC, essentially having to re-implement what clang presumably does (and LLVM not being tied to C[++] like gcc is). The user then overriding the low-level LLVM calling convention with the UDA then may mess up everything.

But I guess most of these few LLVM-supported calling conventions decay to the normal C calling convention for non-exotic targets, excl. 32-bit x86 with its jungle of calling conventions.

Mar 17 '25 15:03 kinke

Hmm, well, I'm going to use fastcc because the alternative is much more trouble, and you already seem to have done the work! I reckon it'd be a cool move to error if an invalid value is given since the names are not what you expect ;)

Mar 17 '25 15:03 TurkeyMan

FWIW, that's all in here: https://github.com/ldc-developers/ldc/blob/61c11b7dafa24d9c67011d03a0b63d3287791c10/gen/uda.cpp#L385-L479

Mar 17 '25 15:03 kinke

Erm...

          .Case("fastcall", llvm::CallingConv::X86_FastCall)
          ...
          .Case("fastcc", llvm::CallingConv::Fast) 
          ...
          .Case("x86_fastcallcc", llvm::CallingConv::X86_FastCall)

...so, was fastcall (which I tried originally) supposed to work? Maybe this map isn't quite right?

Mar 17 '25 16:03 TurkeyMan

And, it actually looks like X86_FastCall is actually what I want here... even though fastcc does work, the spec is not as precise. Not clear why fastcall didn't actually work? It looks like it is a valid name?

Mar 17 '25 16:03 TurkeyMan

Yeah I was surprised too. These x86 enum values aren't in the LLVM langref. Feel free to check the LLVM or clang source to get to the bottom of it. ;)

Mar 17 '25 16:03 kinke