This is the tracking issue for RFC 2627, #[link(kind="raw-dylib")].

Opens:

• [x] Implementation for Windows (top priority) (msvc: #84171) (gnu: #90782)
  • [x] Emitting idata sections
  • [x] New link_ordinal attribute (#89025)
  • [x] Testing in winapi/windows-rs with nightly Rust before stabilization (https://github.com/microsoft/windows-rs/pull/977 and https://github.com/microsoft/windows-samples-rs/pull/18)
  • [ ] Supporting all calling conventions
    • [x] cdecl
    • [x] stdcall
    • [x] fastcall
    • [ ] vectorcall
• [ ] Tests, including all the corner cases mentioned in the reference-level explanation
• [x] Documentation (https://github.com/rust-lang/rust/pull/87315)
• [ ] Implementation for Linux and other platforms
• [ ] Implementation of a pure Rust target for Windows (no libc, no msvc, no mingw). This may require another RFC

cc https://github.com/rust-lang/rfcs/issues/1061

I don't think I have the expertise to implement this feature, but I could write some (currently failing) tests for use by the implementor?

(Background, Ran into this exact problem linking mscorlib.dll into executables. Had to write a C++ level wrapper to get it to work.)

A very first PR has landed, now the link_ordinal attribute is added. Want to say thank you to @Centril for helping me on this~

There's now a bounty for this issue to incentivize someone to work on implementing this feature.

https://www.bountysource.com/issues/70533351-tracking-issue-for-rfc-2627-link-kind-raw-dylib

Some information I've gathered:

• The spec can be found here. It's a bit hard to understand in isolation as it's incomplete and some numbers are given in bits and others in bytes. You need to figure this out from other resources (see below).
• We ONLY have to worry about the .idata section. Despite the syntax suggesting the opposite, the information is purely additive and doesn't change ANYTHING about the import itself (additional attribute in LLVM IR, etc). Roughly, .lib files that this feature wants to make unneccessary are condensed idata sections and the linker extracts the relevant ones for us. This is good for us because we don't need LLVM to support any custom attributes. Not tried it, but emitting additional sections should be enough, and there is LLVM API support for adding sections.
• We'll have to generate not a single .idata section but rather multiple .idata$2, .idata$3, .idata$4, .idata$6, .idata$7 ones. See this part of the spec that explains the rationale
• @SlugFiller has given a nice example in the original issue #30027 in terms of llvm IR code what has to be done.
• Basically, one "only" needs to create this IR code via the llvm API wrappers. The place for this is the librustc_codegen_llvm crate. One probably has to invoke the llvm::LLVMSetSection function in some fashion (just giving you something to grep for)
• The current size check as of #63948 for the ordinals is too tolerant. The spec (section "import lookup table") only allocates 16 bits for both PE32 and PE32+.

To my best knowledge, these are the meanings of the .idata$* sections. See the spec/example for further understanding:

section	name in the MS spec	description
.idata$2	Import directory table	main list containing pointers into the other tables. Each entry in the list concerns one dll and contains pointers to the import address/lookup table, dll name table
.idata$3	Import Address Table	At compile time same as import lookup table. Just duplicate the output! Not sure whether it's actually .idata$5
.idata$4	Import Lookup Table	Has pointers to the function names
.idata$6	Hint/Name Table	Contains the names of the functions we want to import. Also contains export name table hints which are NOT the same as ordinals and we'll probably have to set to 0
.idata$7	No name in the MS spec, let's call it DLL name table	Contains the names for the dll files

The main table is the import directory table which contains pointers into the other tables. Linkers are smart enough to assemble all the relative pointers.

As for who's implementing it, I've stopped contributing patches to the compiler years ago. Better someone else does it. Hope that with my help instructions and @retep998 's bounty there will be some movement on this. Of course I'm around to answer questions.

that has been invaluable so far, thank you

Just as a note about prior art, Apple stopped shipping .dylibs in their SDKs a while ago in favor of "TBD files" which are just YAML descriptions of shared libraries with enough information to link against them. Unfortunately they have zero actual documentation about them available, but the Swift compiler seems to support both generating and linking against them: https://github.com/apple/swift/blob/5d8af8ccf55f3d3dd7ad55dd8fbdc97d15c5b6bf/test/TBD/linking-with-tbd.swift https://github.com/apple/swift/blob/5d8af8ccf55f3d3dd7ad55dd8fbdc97d15c5b6bf/lib/FrontendTool/TBD.cpp

Those "TBD files" are just the YAML equivalent of import libraries on Windows and are still separate files from the source code, so it isn't that novel.

For more prior art, Go uses comments. Windows example, Mac OS example.

More prior art: C# has attributes similar to Rust.

The issue description lists an item "Implementation for Linux and other platforms". The RFC doesn't really talk about it. Linux also doesn't support mapping undefined symbols to specific shared objects. Is the intention to let you specify a symbol version on Linux? Or to add a DT_NEEDED item to the dynamic section without needing the corresponding shared object on your filesystem?

Or to add a DT_NEEDED item to the dynamic section without needing the corresponding shared object on your filesystem?

This would actually be useful to me, I'm now generating a fake shared object to link against: https://github.com/fortanix/rust-sgx/commit/b9c29351b1291eb133e2a54d179784445b9ce665#diff-f52ac431696817278bbfebda2cffffd3

@jethrogb The latter; raw-dylib on Linux should allow linking to a specified shared library without having it available at build time. That would simplify building some kinds of -sys crates.

This is unrelated to symbol versioning. In the future, we should have better ways of handling versioned symbols (both exporting and importing), but that doesn't need to happen right away, and it shouldn't block this.

One other benefit of raw-dylib on Linux: it would enable linking to the vDSO, which works like a shared library but doesn't exist on the filesystem to link against.

Yes that's exactly what I'm talking about in https://github.com/rust-lang/rust/issues/58713#issuecomment-703095092.

@jethrogb Ah, sorry, I hadn't followed that link. Yes, that's the exact case I was thinking of. :)

Hello, @joshtriplett and @retep998 . I'm a Windows developer (as in, working on Windows itself, at Microsoft), and I'd like to help advance this RFC.

Is there an existing prototype implementation?

The spec seems to cover all of our requirements. The main thing that is missing is how (or whether) to support delay-binding imports. PE/COFF has tables that describe how to represent delay load import tables (https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#delay-load-import-tables-image-only). It would be great to cover that in the RFC, even if it isn't implemented in a first version of the support for this. I'd be happy to write up text for handling delay-loads, and submit a PR for the RFC.

hey @sivadeilra - i have a draft pr here, but we were unsure that writing the object files directly was the way to go - i was thinking of revamping the pr to generate "short import" .lib files and passing those to the linker back in august, but i've never got around to it

there's more discussion on this over on the zulip topic:

https://rust-lang.zulipchat.com/#narrow/stream/242869-t-compiler.2Fwindows/topic/Tracking.20raw_dylib.20progress.2E

Since this issue is referencing my (old) example, I figure I should post a long-overdue correction to one of my statements, which is repeated here.

In my example, and related communication, I implied that the idata section is part of the PE format, and is the only thing necessary to define DLL imports. This is inaccurate. Windows itself does not, in fact, attribute any special meaning to the idata section. Rather, a pointer within the PE header points at the start of the import table. In practice, this is almost always the start of the idata section, but there's no particular reason why it couldn't be in the middle of the code section.

The real importance of the idata section, is that both MSVC's and MinGW's linkers will detect its presence in object files, and automatically set the PE header's import table pointer to the start of that section, if it is present. This is the reason why that pointer almost universally points at the start of the idata section. The (very) unfortunate (and ironic) exception to this, however, is LLD.

Disclaimer: This following information was gathered years ago, and could very well be outdated. More recent testing is necessary.

LLD does not give special treatment to idata sections inside object files. Instead, it has a special code path for handling files with a "lib" extension. When such a file is provided as input, LLD checks a few markers to see if it's an import library. If it passes the test, LLD actually decompiles the idata section inside it as an import table, into its own internal data structure. After going over all the files, LLD recompiles the import data it gathered into its own original idata section, and sets the import table pointer to that section. If LLD's internal import data structure is empty, it simply sets the import table pointer to NULL, even if an idata section exists in the object code. At the time I've looked at its code (several years ago) this was the only way to set the import table pointer in the PE header.

Obviously, this method is horribly convoluted, and hopefully, someone at the LLD dev team has realized this and corrected it in the years that have passed since I've last checked. If not, better late than never.

In the mean time, a usable workaround is to produce a lib file instead of an obj file, essentially generating custom import libraries, similar to the dll tools available in the MSVC toolchain. I haven't tested this, and it's been years since I've looked at LLD's code, but it might be as simple as renaming the "obj" file to "lib". Do note that if LLD does detect that a "lib" file is an import library, it will discard any code within it, so the import table and code have to be separated into different files.

This makes me wonder: how does gollvm do it? Do they generate custom idata sections? They support linking with various linkers iirc.

Also. The Zig programming language can cross-compile. It might be worth looking at that.

I'm looking into implementing this support, but I'm still very much investigating and learning about the problem. Do folks mind if I assign this issue to myself while I do, or should I wait until I'm closer to implementation?

@rustbot claim

@ricobbe ive been working on a lot of it, and id love to help out if i can - ive tried a bunch of different approaches, and never quite got a working solution

id love to chat about it if possible

Just failing with a compile error if on non windows platforms seems best for now – adding support for whatever “raw_dylib” means on Linux / mac can happen later

On Tue, Mar 16, 2021 at 04:27:40PM -0700, Richard Cobbe wrote:

I think I'm closing in on an implementation for the simple case (Windows-only, no support for #[link_name] or #[link_ordinal]), and I don't see significant obstacles to supporting those two attributes.

I do have one question that I don't see addressed in the RFC, however: should rustc allow #[link(.., kind = "raw-dylib")] on non-Windows platforms? Or is the assumption that we'll also implement Linux support as described in the "Future possibilities" section of the RFC, making this question moot? (And if so, what about MacOS?) I'd like to get support for this feature on Windows complete without gating it behind Linux and MacOS (as indicated by the RFC), as this unblocks other things we're working on, but I'm willing to discuss this point.

Implementing Windows first seems fine; that's the platform that needs it most.

Current state of this work:

• we emit idata sections for lib crates
• we support cdecl, stdcall, and fastcall, but not vectorcall
• @kennykerr is starting to use the feature in the windows crate under a feature flag

We believe the work has progressed far enough for the winapi crate to begin using it under a feature flag.

As the work so far provides a lot of functionality for Rust devs trying to interoperate with the Windows API, we'd like to split this off as a separate feature and begin stabilization. There's obviously some additional work that needs to happen on our end before this is possible, primarily ensuring good test coverage and documenting the work completed so far in the unstable book. I'll start working on those items today/early next week.

the major missing feature is ordinal imports, right?

the major missing feature is ordinal imports, right?

That's correct, yes.

It works marvelously. Ship it! 😎

@ricobbe I remember the PR which added initial support only added support for the proprietary msvc toolchain. Does it work with the GNU toolchain yet? Does it work with LLD?

@est31 it works for windows-gnu when using LLD but doesn't work with the default BFD linker.

PR for addition to unstable book: #87315

@retep998

Not sure if you've been following this issue, so I just wanted to make sure you were aware that "raw-dylib" should be usable in the winapi crate, to get some real-world experience with the feature. I'd love to hear about any issues you encounter!

I have a question about one of the requirements in the RFC, in the second paragraph of the "Motivation" section. Specifically, which of the two options below is the use case we care about?

1. A crate needs to call function f from library A.DLL and function g from library B.DLL. However, B.DLL also exports a function named f, and we need to avoid confusion during linking.
2. A crate needs to call function f from library A.DLL and function f from library B.DLL.

IIUC, the MSVC toolchain doesn't support use case 2 even when working entirely in C, although I haven't verified that yet myself. Figured I'd clarify the requirements before digging into that in more detail.

@ricobbe This RFC supports both situations via #[link_ordinal] and #[link_name] attributes. So, to answer your question,

mod all_imports_together {

#[link(name = "a.dll", kind = "raw-dylib")]
extern "system" {
    #[link_name("f")] fn a_f() -> i32;
}

#[link(name = "b.dll", kind = "raw-dylib")]
extern "system" {
    #[link_name("f")] fn b_f() -> i32;
}
}

However, it will be more realistic to put different imports into different modules ¯\_(ツ)_/¯

mod a {
#[link(name = "a.dll", kind = "raw-dylib")]
extern "system" {
    fn f() -> i32;
}
}

mod b {
#[link(name = "b.dll", kind = "raw-dylib")]
extern "system" {
    fn f() -> i32;
    fn g() -> i32;
}
}

fn main() {
  a::f();
  b::f();
}

Similar, in Delphi it is separated on a language level as well:

function FunctionName(): Result; stdcall; external 'name.dll' index 1;

function MessageBox(
    hWnd: HWND; 
    lpText: PWideChar;
    lpCaption: PWideChar; 
    uType: UINT
 ): Integer; stdcall; external 'user32.dll' name 'MessageBoxW';

As you see, they support both ordinal and (re)named imports.

I have a question about one of the requirements in the RFC, in the second paragraph of the "Motivation" section. Specifically, which of the two options below is the use case we care about?

1. A crate needs to call function `f` from library A.DLL and function `g` from library B.DLL.  However, B.DLL _also_ exports a function named `f`, and we need to avoid confusion during linking.

2. A crate needs to call function `f` from library A.DLL _and_ function `f` from library B.DLL.

IIUC, the MSVC toolchain doesn't support use case 2 even when working entirely in C, although I haven't verified that yet myself. Figured I'd clarify the requirements before digging into that in more detail.

Use case 2 should be supportable with this feature as specified in the RFC. Import libraries are a mapping from a symbol name to a symbol name or ordinal plus a dll (HashMap<Symbol, (SymbolOrOrdinal, Dll)>). Normally in the C world this would look like "_foo", -> ("foo", "bar.dll") or similar, however in Rust we can make the key a mangled Rust symbol! As a result we can have two unique keys for two symbols, that resolve to symbols with the same name but different dlls!

Use case 2 should be supportable with this feature as specified in the RFC. Import libraries are a mapping from a symbol name to a symbol name or ordinal plus a dll (HashMap<Symbol, (SymbolOrOrdinal, Dll)>). Normally in the C world this would look like "_foo", -> ("foo", "bar.dll") or similar, however in Rust we can make the key a mangled Rust symbol! As a result we can have two unique keys for two symbols, that resolve to symbols with the same name but different dlls!

It's not clear that the MSVC linker supports this in practice. I tried something very close to the second example from @pravic's most recent comment (omitting only the declaration of g); the implementation of the two functions f in the various DLLs basically just printed "f from A.DLL" and "f from B.DLL". While I expected to see A.DLL's f and B.DLL's f called once each, in that order, I actually saw A.DLL's f called twice.

I'd like to try an equivalent example written entirely in C, to determine if the error is in rustc or MSVC. However, I don't know of a way to declare two separate functions, both named f, that come from different libraries, without getting a "function redefined" error. I don't think MSVC has anything equivalent to the #[link_name] attribute, but please let me know if I'm wrong.

If you'd like to take a look at what I tried, in case I omitted anything, you can find it here on my github fork.

It might be possible to achieve the desired results by using #[link-ordinal], but we haven't implemented that functionality yet.

@ricobbe I believe if you create the import tables in LLVM similar to the example I gave in #30027, then link with MSVC's linker, it should work. The example can easily be extended to two DLLs simply by addin another descriptor to the .idata$2 section, before the null descriptor. It should also work with MinGW's linker. However, I would repeat the above note that, based on my previous test, it does not work with LLVM's native linker. What more, assuming LLVM's native linker still treats libraries the same way, it probably couldn't work, as it wouldn't allow custom name mangling.

P.S. It might be worthwhile to use a hex editor, or perhaps a small utility script, to read the PE header on the final executable, and see that the import table looks right. That's how I figured out LLVM's linker was leaving the import table pointer as null, instead of pointing it to the .idata section.

Is this functionality (multiple libraries that export the same symbol) necessary in order to stabilize the work as already completed?

I'd argue that, even without this functionality, current support for raw-dylib enables a large number of important use cases, such as interoperability with the Windows API. Also, since it doesn't appear that C and C++ even allow programmers to write programs that attempt to use symbols of the same name from different libraries, wouldn't the current implementation be sufficient in many cases when interoperating with DLLs intended for use by C and C++ programs?

The conversation around stabilization seems to have stalled. Is there anything outstanding that would prevent moving the functionality that's already been implemented into stabilization?

@joshtriplett

As a side note: it doesn't look like I'm going to be able to devote much time or energy towards addressing the remaining functionality, as we believe that the work done so far handles our use case (interoperability between Rust and the Windows APIs). Would anyone mind if I removed myself as the person assigned to this issue?

And to be explicit about our commitment: we do expect issues with the completed work to arise during stabilization as we get real-world experience working with the feature, and I certainly intend to take the lead on helping to resolve those. It's just a question of who will drive implementing the remaining functionality in the RFC.

Right now, this feature is only partially supported on the *-windows-gnu target family, which has several targets listed as tier 1. In order to use the feature, users of those targets have use the LLD linker, which is not the default. LLD is really awesome, but the manual step needed to use LLD would make the setup story for users of those targets harder. Maybe this can be resolved by progressing on the LLD rollout plan #39915 for *-windows-gnu targets?

There is also a lack of *-windows-gnu based tests. I can't find a test that checks that raw-dylib works with the *-windows-gnu target together with LLD. For example, this test still has only-windows-msvc. There is a test for a warning warning for the gnu targets, that says the feature doesn't work there. Is it turned off when LLD is in use?

If I had to name an issue blocking stabilization, it would be better *-windows-gnu support:

• Tests in the rustc test suite that check that *-windows-gnu works with the feature when using LLD
• Removal of the warning when LLD is used, rewording it to mention LLD as an option, as well as tests for that
• Ideally: Adoption of LLD as default *-windows-gnu linker (#39915) OR better support in the BFD linker (unlikely, but mentioning that this is also an option). At the very least there should be good docs for how to set LLD as the linker.

@est31

Right now, this feature is only partially supported on the *-windows-gnu target family, which has several targets listed as tier 1. In order to use the feature, users of those targets have use the LLD linker, which is not the default.

The raw-dylib feature as currently implemented deliberately does not support the *-windows-gnu targets, and so the only relevant test is a UI test to ensure that we issue a diagnostic upon encountering raw-dylib on those targets.

I did explore the possibility of using LLD, but @mati865 informed me (in a private communication on April 29) that LLD was not supported in automated testing for rustc. I'm deeply reluctant to claim to support a feature that cannot be tested in automation. If that restriction is no longer in effect, then I'm willing to investigate using it to support this functionality.

I agree that officially supporting something without tests isn't good. As for LLD support in the test suite, I wonder how it's solved for wasm targets, because IIRC there, only LLD is being used. Eventually as rustc migrates to LLD, this problem has to be solved anyways, no?

Eventually as rustc migrates to LLD, this problem has to be solved anyways, no?

I assume so, yeah. I think the important question for now, however, is whether solving the LLD-in-automation problem should be required in order to stabilize this work for the windows-msvc platforms.

LLD as the default windows-gnu linker has big question mark next to it: BFD. LLD works fine with all kinds of import libraries but emits only one type, the one that BFD has issues with. That means we cannot guarantee that dynamic libraries will be linkable by platform default linker when defaulting to LLD.

BTW I hope to add purely LLVM based mingw-w64 targets to Rust but haven't got the time to work on upstreaming it (x86_64 has been already well tested within MSYS2). It could act as a confirmation that LLD works fine for this feature.

Has anyone already looked into this:

Implementation of a pure Rust target for Windows (no libc, no msvc, no mingw). This may require another RFC

I would imagine that we have to create a new target e.g. x86_64-pc-windows-raw and for that target link against all of the Win32 functions in std with kind="raw-dylib". We than probably also have to default to rust-lld and implement everything that happens before main e.g. exe_common.ini and the rest of the vcruntime ourselves. Is there something I forgot? I did parts of this, while building a toy no_std raw-dylib Hello World which compiles without problems under linux (without mingw or msvc, just pure Rust).

Status update: as before, we'd love to see this feature taken up by the winapi and windows-rs crates, to get some real-world experience with it. Since my last message, I've learned that many users of these crates run the Rust toolchain on Linux to produce Windows binaries, and that the windows-gnu target is crucial for these users. So I'm trying to understand exactly why the current implementation fails for windows-gnu builds, and what it would take to fix this.

What I know so far:

• running the .EXE produced by running src/test/run-make/raw-dylib-c (tweaked slightly to run on windows-gnu) causes an access violation upon trying to call one of the functions imported from a raw-dylib DLL.
• I've used llvm-dlltool to produce an import library from a .DEF file, and used the MSys2 toolchain to compile and link a C program that calls a function from this library. The resulting executable runs and produces the expected results. So the earlier theory, that binutils can't read LLVM's import libraries, doesn't seem to be the whole story.

I'm fairly new to dealing with libraries and linking at this low level of abstraction, so figuring out exactly what's going on here is likely to take me some time, but I am continuing to work on it.

I'm a little confused by the last comment, as the big reason you want windows-gnu to build on Linux is to avoid a dependency on the non-redistributable Windows SDK import libraries, the very thing this would resolve - if you copy those files over to Linux and set linker flavor to lld-link you can build windows-msvc fine on Linux. Sure, you could also have issues with native code, but that's not something Rust can fix.

That said, it's not like I'm against having it working on windows-gnu!

I'd been under the impression that, when running for a *-windows-msvc target, rustc actually invokes the MSVC linker to generate the final .EXE/.DLL/.RLIB/.LIB/whatever. Therefore, since Microsoft's LINK.EXE isn't available when running the compiler on Linux, the *-windows-msvc targets are unavailable, and users in this setting have to use *-windows-gnu instead. Is that not correct?

I've used llvm-dlltool to produce an import library from a .DEF file, and used the MSys2 toolchain to compile and link a C program that calls a function from this library. The resulting executable runs and produces the expected results.

I was able to reproduce that and confirmed the import library created by llvm-dlltool is the same as one created by Rust. Maybe the difference is related to who reads the import library? With raw-dylib it's read by add_archive and added to .rlib while with manual linking it's read by ld.exe.

As the test I had used lib.rs without raw-dylib with symbols linked like that: #[link(name = "extern_1")] and had both extern_1.dll and extern_1.dll.a (the import lib) in that directory. It worked fine. Then I used lib.rs with raw-dylib while import libraries were still present and the binary segfaulted.

Symbols in the .rlib:

$ nm /d/Projekty/rust/build/x86_64-pc-windows-gnu/test/run-make/raw-dylib-c/raw-dylib-c/libraw_dylib_test.rlib | rg extern
                 U extern_fn_1
                 U extern_fn_2
                 U extern_fn_3
extern_1.dll:
                 U extern_1_NULL_THUNK_DATA
0000000000000000 I __IMPORT_DESCRIPTOR_extern_1
extern_1.dll:
extern_1.dll:
0000000000000000 I extern_1_NULL_THUNK_DATA
extern_1.dll:
0000000000000000 I __imp_extern_fn_1
                 U __IMPORT_DESCRIPTOR_extern_1
0000000000000000 T extern_fn_1
extern_1.dll:
0000000000000000 I __imp_extern_fn_2
                 U __IMPORT_DESCRIPTOR_extern_1
0000000000000000 T extern_fn_2
extern_2.dll:
                 U extern_2_NULL_THUNK_DATA
0000000000000000 I __IMPORT_DESCRIPTOR_extern_2
extern_2.dll:
extern_2.dll:
0000000000000000 I extern_2_NULL_THUNK_DATA
D:\msys64\mingw64\bin\nm.exe: lib.rmeta: no symbols
extern_2.dll:
0000000000000000 I __imp_extern_fn_3
                 U __IMPORT_DESCRIPTOR_extern_2
0000000000000000 T extern_fn_3

My theory: ld.bfd.exe looks at the .rlib and sees undefined extern_fn_* but also imports for that symbol within the same archive. Here we hit one of the bugs I mentioned earlier (like https://sourceware.org/bugzilla/show_bug.cgi?id=25374). Somehow when they are in separate archives this works fine for this case. I'm not sure we can safely rely on that though.

I'd been under the impression that, when running for a *-windows-msvc target, rustc actually invokes the MSVC linker to generate the final .EXE/.DLL/.RLIB/.LIB/whatever. Therefore, since Microsoft's LINK.EXE isn't available when running the compiler on Linux, the *-windows-msvc targets are unavailable, and users in this setting have to use *-windows-gnu instead. Is that not correct?

It's possible to use the -windows-msvc target on linux, but one has to either use LLVM's "MSVC emulation" programs (lld-link, clang-cl and others), or run the real MSVC programs in wine. I personally do the former, and there's a recent article explaining how to set it up: https://jake-shadle.github.io/xwin/.

However, I don't think most people do that. The "usual" way to cross-compile windows binaries from linux is to install the MinGW toolchain on linux (for instance, on ubuntu, apt install gcc-mingw-w64), and use that toolchain to build with the -windows-gnu target.

raw-dylib, combined with a move to use the lld linker by default, would make cross-compiling with the -msvc toolchain much easier.

With that said, -windows-gnu would still benefit from this work. The gnu and MSVC toolchain have some small ABI differences (notably, in how unwinding works) that may force a project to use the -gnu target over the -msvc target (for instance, if trying to link a proprietary lib that only has a mingw variant). If raw-dylib is to be used unconditionally in winapi/windows-rs, then it'll need to work for the -gnu target, or a lot of people will end up being unable to use those foundational libraries.

Opened https://github.com/rust-lang/rust/pull/88801 as proof of concept how this can be fixed for MinGW targets without using LLD or fixing BFD. I'm a bit time limited right now so it will take some time to make it ready.

Remember that pc-windows-msvc also requires the CRT to link against. There are bits of the CRT that have to be linked in statically (even when using the dll version of the CRT), so raw-dylib does not save you there. You'll either need to copy over those CRT runtime bits from a VC++ installation, or use MinGW's via pc-windows-gnu.

Rewriting things to not need those CRT bits is the entire point of this:

Implementation of a pure Rust target for Windows (no libc, no msvc, no mingw). This may require another RFC

Would it be possible for std lib to start using "raw-dylib" for its own Windows dll imports? Or would there be issues with that?

Would it be possible for std lib to start using "raw-dylib" for its own Windows dll imports? Or would there be issues with that?

This would probably be the first step in this direction:

Implementation of a pure Rust target for Windows (no libc, no msvc, no mingw). This may require another RFC

The only thing left then would be implementing standalone VS CRT initialization code.

For sure but I mean if I, hypothetically, submitted a PR today that changed all Windows imports to use raw-dylib, would anything be expected to break? Is it too early?

The only thing left then would be implementing standalone VS CRT initialization code.

IIRC, it's quite possible that the VS CRT could be made open source later this year. If that does happen (and depending on the license) it may make things easier.

For sure but I mean if I, hypothetically, submitted a PR today that changed all Windows imports to use raw-dylib, would anything be expected to break? Is it too early?

I think that would make mingw builds start to have a hard dependency on dlltool. I think it is normally available, but I can imagine it isn't available in some cases.

Note that at least MSVC target has a hard dependency of SEH facility provided by MSVC C runtime library - for more information, go to check seh.rs in the panic_unwind library.

Specifically, for those that don't want to look, for MSVC targets LLVM always emits the exception personality as the CRT implemented _CxxFrameHandler3.

I suppose you should be able to manually implement that in rust, if you're some kind of complete lunatic?

Specifically, for those that don't want to look, for MSVC targets LLVM always emits the exception personality as the CRT implemented _CxxFrameHandler3.

I suppose you should be able to manually implement that in rust, if you're some kind of complete lunatic?

Required functions actually are more than just _CxxFrameHandler3 : Look at this import info, which is about the released version of most recent ripgrep built on MSVC target. Except those mem* functions, all remaining functions imported from VCRUNTIME140.dll are exception handling related.

I believe windows-gnu implementation is incomplete: https://github.com/rust-lang/rust/pull/90782#issuecomment-1011267153 This should work for building std as part of rustc since we build it with full C toolchain but for any crate that wants to rebuild std or reuse its code that would be problem.

For sure but I mean if I, hypothetically, submitted a PR today that changed all Windows imports to use raw-dylib, would anything be expected to break? Is it too early?

I think that would make mingw builds start to have a hard dependency on dlltool. I think it is normally available, but I can imagine it isn't available in some cases.

Dlltool is part of binutils so one can expect it to always exist.

Required functions actually are more than just _CxxFrameHandler3 :

Hmm, what would be the implication of getting LLVM to emit a RustFrameHandler and RustThrowException, etc? It seems like Windows only cares about you using SEH, and throwing across FFI is already undefined, right?

Probably something nightmarish, it tends that way in these areas.

Also note #93842.

I've discovered a problem with how this feature, as currently implemented, supports stdcall functions from certain DLLs on i686-pc-windows-msvc. (I don't yet know if this problem also affects i686-pc-windows-gnu; I'll try to reproduce it there in the next couple of hours.) The details are included below, but in summary, it appears that the import libraries generated by rustc/LLVM only work for DLLs that are linked in a specific way, such as with the DLL in src/tests/run-make/raw-dylib-alt-calling-convention. For other DLLs, rustc generates an .EXE file that does not load -- in the example below, running cargo +nightly-i686-pc-windows-msvc run results in a dialog box with the error message, "The procedure entry point _WaitForSingleObject@8 could not be located in the dynamic link library ...\path\to.exe."

[EDIT: this problem also occurs on i686-pc-windows-gnu.]

As near as I can determine, the difference is in how the two DLLs (kernel32.dll and the one in the rustc test) specify the symbols to be exported. My understanding is that kernel32.dll does not use __declspec(dllexport) and instead supplies a .DEF file to the linker when creating kernel32.dll; this .DEF file lists the exported symbols. Possibly relevant is that the .DEF file uses the bare names, not the names mangled to match the convention for stdcall functions -- that is, the .DEF file specifies WaitForSingleObject, not _WaitForSingleObject@8 -- so we can use the same .DEF file regardless of target architecture.

On the other hand, when I constructed the raw-dylib-alt-calling-conventions test, I was unaware of this, so I specified __declspec(dllexport) on each function and did not provide a .DEF file. If I switch the test case to match the kernel32.dll model, I get an equivalent error message.

Linking against kernel32.dll is an important use case, so I'm investigating this. I'm beginning to get a sense of what is "wrong" with the import library that LLVM generates, although unfortunately LLVM's writeImportLibrary function does some unexpected transformations to its arguments before creating the library, which makes it slow (but not impossible) to test my current theories.

I'm hoping to achieve 2 things with this comment. First, if anyone has any suggestions on how to get LLVM to do what we want here, I'd love to hear them. Second, it looks like we need to have a conversation about how the raw-dylib feature interacts with what I expect to be a wide variety of DLLs in existence. If it turns out that the way I'm building the DLL in raw-dylib-alt-calling-convention is Just Wrong and We Never Do That, and we decide not to support that case, then (assuming we can figure out how to get LLVM to do what we want) I'm happy to update the test case and declare victory. If not, and if the community feels that we need to support both kinds of DLLs, then we may need to consider adding additional information to the Rust declarations, possibly with a new attribute, to communicate to the raw-dylib feature what kind of linking to use, since I don't think it's possible for the compiler to determine that with the information currently available to it.

Test case:

Cargo.toml:

[package]
name = "stdcall-example"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[lib]
name = "stdcall_lib"
path = "src/lib.rs"

[[bin]]
name = "stdcall_bin"
path = "src/main.rs"

[dependencies]

src/lib.rs:

#![feature(raw_dylib)]
#[link(name = "kernel32", kind = "raw-dylib")]
extern "system" {
    fn WaitForSingleObject(handle: isize, dwmilliseconds: u32) -> u32;
}
pub fn lib_main() {
    unsafe { WaitForSingleObject(0, 0); }
    println!("ok");
}

src/main.rs

use stdcall_lib::lib_main;

pub fn main() {
    lib_main();
}

The program is supposed to print ok and terminate cleanly.

@ricobbe if you don't want to use the mangled name on the symbol import, you can use the #[no_mangle] or #[link_name = "..."] attribute on the fn item.

if you don't want to use the mangled name on the symbol import, you can use the #[no_mangle] or #[link_name = "..."] attribute on the fn item.

I'll give that a try, thanks!

EDIT: unfortunately, neither attribute had any effect -- and using #[no_mangle] produced a compiler warning stating that this attribute has no effect on a foreign function.

That said, I'm not sure I ever implemented support for #[link_name] when used with raw_dylib, so I can investigate that. That needs to happen anyway, and it'll probably be less effort than trying out changes to writeImportLibrary, if for no other reason than it means I can avoid the long LLVM compile times.

FTR mingw-w64 specifies WaitForSingleObject@8 (leading _ is automatically added on 32-bit platforms) in the .def file: https://gitlab.com/mati865/mingw-w64-mirror/-/blob/master/mingw-w64-crt/lib32/kernel32.def#L1543 This works with both GNU and LLVM toolchains (at least in MinGW mode). When generating import library some magic takes place and IIUC binaries can use symbols with or without dllimport:

$ nm /clang32/lib/libkernel32.a | grep _WaitForSingleObject@
00000000 T _WaitForSingleObject@8
00000000 T __imp__WaitForSingleObject@8

FTR mingw-w64 specifies WaitForSingleObject@8 (leading _ is automatically added on 32-bit platforms) in the .def file: https://gitlab.com/mati865/mingw-w64-mirror/-/blob/master/mingw-w64-crt/lib32/kernel32.def#L1543 This works with both GNU and LLVM toolchains (at least in MinGW mode). When generating import library some magic takes place and IIUC binaries can use symbols with or without dllimport:

I'm not entirely sure I follow this, unless you're saying that the way I'm building the DLL in src/test/run-make/raw-dylib-alt-calling-convention shouldn't be supported.

Also, I think we may have gotten wires crossed slightly -- the question is not whether or not the calling module uses __declspec(dllimport); the question is whether (among other things) the defining module uses __declspec(dllexport). And, IIUC, if the #[link] attribute is present, rustc now always performs the optimization that, in MSVC, is enabled by __declspec(dllimport), since the change in #84758.

Oh, sorry I have misunderstood it.

My understanding is that kernel32.dll does not use __declspec(dllexport) and instead supplies a .DEF file to the linker when creating kernel32.dll; this .DEF file lists the exported symbols. Possibly relevant is that the .DEF file uses the bare names, not the names mangled to match the convention for stdcall functions -- that is, the .DEF file specifies WaitForSingleObject, not _WaitForSingleObject@8 -- so we can use the same .DEF file regardless of target architecture.

Unfortunately I have zero knowledge about MSVC but I think regardless of using dllexport attribute or .def file the select symbols are exported the same way.

the question is whether (among other things) the defining module uses __declspec(dllexport). And, IIUC, if the #[link] attribute is present, rustc now always performs the optimization that, in MSVC, is enabled by __declspec(dllimport), since the change in #84758.

Judging by llvm-readobj --coff-exports /c/windows/SysWOW64/kernel32.dll it exports the symbols just like any other DLL:

...
Export {
  Ordinal: 1523
  Name: WaitForSingleObject
  RVA: 0x23A20
}
...

Note: this tool skips decoration (@ <number of bytes in arguments>).

So from my limited knowledge one should use dllimport attribute to get these symbols and rustc does the right thing.

FWIW, Windows binaries always use no_mangle or equivalent when exporting from a DLL.

FWIW, Windows binaries always use no_mangle or equivalent when exporting from a DLL.

It depends what you mean by "Windows binaries". Standard Windows API dlls? Yes, they do not use mangling - either ordinals or "clean names".

DLLs in general? They are free to use whatever they want. Different languages and different toolchains use different mangling.

@mati865

Unfortunately I have zero knowledge about MSVC but I think regardless of using dllexport attribute or .def file the select symbols are exported the same way.

Much to my chagrin, that turns out not to be the case, as demonstrated by the following example. (To my knowledge, this difference only arises for __stdcall functions on 32-bit Intel; 32-bit cdecl and 64-bit are both fine. That said, I haven't tried fastcall or vectorcall yet.)

dllexport.c:

__declspec(dllexport) int __stdcall exported_function(int x) {
    return x * 2;
}

def-file.c:

int __stdcall exported_function(int x) {
    return x * 2;
}

def-file.def:

LIBRARY def-file
EXPORTS
    exported_function

I compiled these in a Developer Command Prompt for VS 2019 (32-bit) shell:

cl /LD dllexport.c
cl /LD def-file.c def-file.def

which produced (among other files) dllexport.dll and def-file.dll. Dumping the export table from both dlls does indicate a difference:

E:\exploration\dll-exports\dllexport>link /dump /exports dllexport.dll
Microsoft (R) COFF/PE Dumper Version 14.29.30140.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file dllexport.dll

File Type: DLL

  Section contains the following exports for dllexport.dll

    00000000 characteristics
    FFFFFFFF time date stamp
        0.00 version
           1 ordinal base
           1 number of functions
           1 number of names

    ordinal hint RVA      name

          1    0 00001000 _exported_function@4

and

E:\exploration\dll-exports\def-file>link /dump /exports def-file.dll
Microsoft (R) COFF/PE Dumper Version 14.29.30140.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file def-file.dll

File Type: DLL

  Section contains the following exports for def-file.dll

    00000000 characteristics
    FFFFFFFF time date stamp
        0.00 version
           1 ordinal base
           1 number of functions
           1 number of names

    ordinal hint RVA      name

          1    0 00001000 exported_function

Note that dllexport.dll exports the symbol _exported_function@4, but def-file.dll exports the symbol exported_function.

I have it on good authority that the 32-bit version of Windows's kernel32.dll is built in the same manner as def-file.dll above. My test of raw-dylib with stdcall (in src/test/run-make/raw-dylib-alt-calling-convention), however, builds its dll in the same manner as dllexport.dll. As demonstrated by the test in my comment from yesterday, rustc's current means of generating the import library doesn't work with kernel32-like libraries (i.e., those built without __declspec(dllexport) and with a .def file). Since raw-dylib-alt-calling-convention passes (at least on my machine, anyway), it seems that rustc's current import library does work with __declspec(dllexport) libraries.

I'll cheerfully admit that I'm fairly new to a lot of this stuff, so it's entirely possible that I'm missing something and the cause of the failure is elsewhere. I do think it unlikely, though, given the difference between the DLLs' export tables, and the fact that the error message refers to a symbol exported by one DLL and not by the other.

Update: I've found a way to invoke LLVM's writeImportLibrary in a way that works with libraries like kernel32.dll, so I think the primary technical hurdle here has been overcome. However, this leaves us with the questions I raised initially:

1. Do we want rustc to be able to link against DLLs generated in both of these two ways?
2. If so, what mechanism do we want to introduce to allow the user to indicate in the .rs file whether the DLL exports the decorated or undecorated names?

My hunch is that the answer to question 1 is "yes." For question 2, do we want to use #[link_name] for this purpose? If so, do we want to make the decorated or undecorated names the default? I'd suggest making the decorated names default in this case, to save users the trouble of working out (or looking up) the argument list sizes.

Alternatively, since it's a binary choice, and since rustc is capable of synthesizing both names, I'd suggest that for usability reasons we should provide a different attribute (say, #[undecorated_name]) that's just a binary switch, to save people the hassle of having to type the function name twice, as in the following:

#[link_name = "ReallyLongExportedFunctionNameThatIsOverlyDescriptiveButHereWeAre"]
fn ReallyLongExportedFunctionNameThatIsOverlyDescriptiveButHereWeAre(x: u32) -> u32

So turns out that MSVC, GCC+Binutils and Clang+LLD targetting mingw-w64 all handle this differently:

GCC+Bintuils:

$ gcc dllexport.c -shared -o dllexport.dll

$ gcc def-file.c def-file.def -shared -o def-file.dll
D:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/11.2.0/../../../../i686-w64-mingw32/bin/ld.exe: warning: resolving _exported_function by linking to _exported_function@4
Use --enable-stdcall-fixup to disable these warnings
Use --disable-stdcall-fixup to disable these fixups

$ nm dllexport.dll | grep export
681c14d0 T _exported_function@4

$ nm def-file.dll | grep export
693c14d0 T _exported_function
693c14d0 T _exported_function@4

Clang+LLD with i686-pc-windows-gnu target:

$ clang dllexport.c -shared -o dllexport.dll

$ clang def-file.c def-file.def -shared -o def-file.dll

$ nm dllexport.dll | grep export
100013b0 T _exported_function@4

$ nm def-file.dll | grep export
100013b0 T _exported_function@4

Do we want rustc to be able to link against DLLs generated in both of these two ways?

I think one the main selling points for raw-dylib was to use it for system libraries which would imply the answer "yes".

If so, what mechanism do we want to introduce to allow the user to indicate in the .rs file whether the DLL exports the decorated or undecorated names?

I worry this might need larger discussion than this thread. I think you could start the topic on Zulip.

For what it's worth, the import library format supports 4 name types https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#import-name-type

The only one I think hasn't been discussed is IMPORT_NAME_NOPREFIX (which is like undecorating the name but only applies to the prefix). Though to be honest I'm not sure if anything uses that but I thought I'd bring it up in case.

For what it's worth, the import library format supports 4 name types

Well, that suggests another possibility for a user-provided annotation. I haven't yet seen a case where IMPORT_NAME_NOPREFIX would be useful, but it would be easy enough to implement, so we might as well.

Just for completeness, I'll point out that we already have a mechanism for dealing with IMPORT_ORDINAL, the #[link_ordinal] attribute. There is test coverage of this feature in src/test/run-make/raw-dylib-stdcall-ordinal. Further, at least in my understanding of how export-by-ordinal works, one must specify a .DEF file when linking such a DLL, and in any case the actual symbol names don't matter in this case. (If that turns out to be incorrect, please let me know -- the sooner I find out about that the faster I can start working on a solution.)

Hm, both short and long forms support ordinals. However, I don't know if LLVM can produce such import libraries or if mingw can consume them.

But yes, in the ordinal case the symbol names only need to be consistent between the application and import library. It doesn't matter what the DLL symbol is.

both short and long forms support ordinals. However, I don't know if LLVM can produce such import libraries or if mingw can consume them.

I think there are two separate issues here: the question of correctly generating import libraries for symbols that are exported via ordinal, and the question of compatibility of import libraries between LLVM and mingw.

For the first question: to the best of my knowledge, support for ordinals is complete. You can find test coverage of it in src/test/run-make/raw-dylib-{stdcall,link}-ordinal, and those tests are passing (for me, at least), on {i686,x86_64}-pc-windows-{msvc,gnu}. (Well, the stdcall tests only run on i686, but you know what I mean.)

For the second: we have established pretty clearly that mingw/binutils does not work with the import libraries that LLVM produces; it generates .EXEs that AV when attempting to call an imported function. So, for -windows-gnu targets, I've adopted the solution of writing a temporary .DEF file and calling out to binutils's 'dlltool' utility to generate the import libraries. This is implemented on master. I'll cheerfully admit that this is a pretty krufty hack, but it demonstrably works -- the various test cases that I've created for raw-dylib do pass on {i686,x86_64}-pc-windows-gnu, at least when I run them manually on my machine. (I'm a little unclear on how much test coverage the rustc CI provides for windows targets, so I won't speak to that.)

the various test cases that I've created for raw-dylib do pass on {i686,x86_64}-pc-windows-gnu, at least when I run them manually on my machine. (I'm a little unclear on how much test coverage the rustc CI provides for windows targets, so I won't speak to that.)

As a Tier 1 targets all of their tests (unless explicitly ignored) have to pass for every PR. So the implementation itself is tested.

I've posted a proposal for the attribute changes to the t-compiler channel on Zulip. Feedback welcome.

Quick note to let folks know that I'll be leaving Microsoft effective tomorrow (April 29) and thus need to hand this project over to others. I've created issues for the outstanding issues that I'm aware of: #93842 and #96534.

It's not yet clear how much time and effort I'll be able to spend on this project going forward; it probably won't be much. I'm also not entirely sure if I'll continue to have access to this particular github account going forward, due to MSFT corporate security policies, although I have at least added a non-MSFT email address to the account, so we'll see.

Thanks for your work on this @ricobbe! Good luck with your new role 😃

Regarding the last "pure Rust target for Windows" bit, I wrote up a summary of things I think we need to do to remove VCRuntime dependency: https://internals.rust-lang.org/t/pre-rfc-remove-rusts-dependency-on-visual-studio-in-4-complex-steps/16708.

This does not remove our dependency on Windows SDK but it's not under Visual Studio licence so it's a lesser issue. I think removing VCRuntime does not require #[link(kind="raw-dylib")] but removing Windows SDK of course does.

Tagging as impl-incomplete based on the checklist in the top comment, but also, a question for the windows-rs folks and others: with the known bugs fixed (e.g. ICEs), would there be value in a partial stabilization even without vectorcall?

a question for the windows-rs folks and others: with the known bugs fixed (e.g. ICEs), would there be value in a partial stabilization even without vectorcall?

Absolutely, only stdcall is essential for windows-rs.

So to be clear, would this now be ready for std's internal use? Last time I asked I think there were outstanding issues.

@rustbot claim

Hi everyone, I'm going to be picking up from where Richard left off on stabilizing this.

It looks like the only two remaining issues are the ones that Richard mentioned above (#93842 and #96534).

Additionally, it seems like there may be some pre-existing incompatibilities between the GNU and LLVM toolchains that are probably out-of-scope for this feature?

Additionally, it seems like there may be some pre-existing incompatibilities between the GNU and LLVM toolchains that are probably out-of-scope for this feature?

It was said this feature has to cover Tier 1 Windows targets so this means MSVC and GNU toolchains. Tier 3 *windows-gnullvm aren't necessary for stabilisation if that's what you mean. Though these targets require basically no changes. I made them generate import lib via LLVM (just like MSVC does) but they could use dlltool just like *-windows-gnu.

With the completion of #98989 this feature is ready to be stabilized for non-x86 platforms (x86_64 and aarch64 for both msvc and gnu environments, it should work on thumbv7a as well, but that's not a tier 1 target).

For x86, we still need two more pieces:

• #99476
• https://github.com/rust-lang/compiler-team/issues/525

Ship it! As soon as that is stabilized, I can turn it on for non-x86 targets in the windows and window-sys crates.

Request for Stabilization

I'd like to stabilize raw-dylib for non-x86 architectures (i.e., x86_64, aarch64 and thumbv7a). A stabilization PR has been submitted as #99916.

Summary

The following two features will be stabilized:

• The raw-dylib kind for the link attribute:

  #[link(name = "exporter", kind = "raw-dylib")]
  extern "stdcall" {
      fn imported_function_stdcall(i: i32);
  }
  

• The link_ordinal attribute:

  #[link(name = "exporter", kind = "raw-dylib")]
  extern "stdcall" {
      #[link_ordinal(15)]
      fn imported_function_stdcall(i: i32);
  }
  

Using either of these features when targeting an x86 architecture will result in an error, unless the raw_dylib feature is enabled.

Documentation

I've submitted the following PRs:

• The Reference: https://github.com/rust-lang/reference/pull/1244

Tests

You can find test cases in:

• run-make/raw-dylib-c - Basic tests of raw-dylib for all architectures, generating both a lib and a bin crate.
• run-make/raw-dylib-alt-calling-convention - x86 specific tests for different calling conventions.
• run-make/raw-dylib-link-ordinal - Basic tests of link_ordinal for all architectures.
• run-make/raw-dylib-stdcall-ordinal - x86 specific tests for link_ordinal using an alternative calling convention.
• ui/rfc-2627-raw-dylib/link-ordinal-and-name.rs - using link_ordinal with link_name results in an error.
• ui/rfc-2627-raw-dylib/link-ordinal-invalid-format.rs - the ordinal provided to link_ordinal must be a number.
• ui/rfc-2627-raw-dylib/link-ordinal-and-name.rsl - using link_ordinal with link_name results in an error.
• ui/rfc-2627-raw-dylib/link-ordinal-missing-argument.rs - an ordinal must be provided to link_ordinal.
• ui/rfc-2627-raw-dylib/link-ordinal-multiple.rs - only one link_ordinal can be applied to a declaration.
• ui/rfc-2627-raw-dylib/link-ordinal-too-large.rs - the ordinal provided to link_ordinal is too large.
• ui/rfc-2627-raw-dylib/link-ordinal-too-many-arguments.rs - link_ordinal only takes a single argument.
• ui/rfc-2627-raw-dylib/multiple-declarations.rs - clashing extern declarations with the raw-dylib link kind must have matching calling conventions.
• ui/rfc-2627-raw-dylib/raw-dylib-windows-only.rs - raw-dylib may only be used when targeting Windows platforms.
• ui/rfc-2627-raw-dylib/unsupported-abi.rs - raw-dylib only supports a subset of calling conventions per architecture.

Unresolved issues

• Test for importing variables: #100073
• Cleanups for the link_ordinal attribute:
  • #100009 (Fixed by #100091 - thanks @chenyukang!)
  • #100069
• Finishing support for x86:
  • Fixing support for vectorcall and adding tests: https://github.com/rust-lang/rust/pull/99476
  • #96534 which is being fixed by #100732

See #100009, I think it would be good to fix before stabilized.

@joshtriplett vectorcall support and tests have been merged in.

@rust-lang/lang & @rust-lang/compiler, do we have a precedent for stabilizing a feature on some CPU architectures but not others?

@michaelwoerister asm! is stable on some CPU archs but not all (only x86/x64, arm/arm64 and riscv).

FYI, remaining non-x86 issues that were identified have been addressed and fixes merged in.

A recent thread on internals noted that we could support this on macOS as well, by generating .tbd files.

I've updated the checkboxes in the issue description to include macOS support.