Failed to build for `aarch64-pc-windows-msvc`

[Berrysoft]

trafficstars

As far as I know, wasmer can target Windows ARM64, and so do cranelift.

https://github.com/bytecodealliance/wasmtime/blob/f242975c49385edafe4f72dfa5f0ff6aae23eda3/crates/runtime/src/traphandlers/windows.rs#L54-L62

[alexcrichton]

This is not currently a supported platform, but unlike https://github.com/bytecodealliance/wasmtime/issues/4434 Cranelift does have an AArch64 backend. I don't believe anyone's tested the Windows support of the backend, however. If you'd like feel free to test locally and we can review patches.

[kpreisser]

Hi there! Are there plans to include Windows ARM64 (AArch64) as supported platform im Wasmtime?

We plan to integrate wasmtime-dotnet into an application that we currently ship for Windows x64/x86/Arm64 and Linux x64/Arm64/Arm32. I understand that the 32-bit archs are not supported by wasmtime/cranelift, but it would be great to have Windows ARM64 support.

Thank you!

[cfallin]

@kpreisser I would say we'd be very interested in contributions for this support, but no one currently involved with the project has the time to work on it, as far as I'm aware.

I actually don't know how much work it would be -- does Windows/aarch64 use a different calling convention, like fastcall on x86-64? It could be the case that it almost works already and just needs a few config changes; I don't know.

There's also the question of testing in CI: we have Windows/x86-64 runners, but does Windows have a "reverse emulator" that can run aarch64 .exes on x86-64, like it can the other way around? If not, I suppose we could treat this the same as macOS/aarch64: support in-tree, but not tested so best-effort.

[jameysharp]

In addition to what Chris said, check out our tiers of support. If someone can get Windows/aarch64 tested in CI then it would be most of the way to tier 2 support. At that point, other contributors can take care of small changes affecting the new target and rely on CI to catch mistakes. Then we just need someone to be available to answer questions about the target and help with occasional bigger changes.

[bjorn3]

Windows on AArch64 has two possible abi's. The first is I believe AAPCS compatible except with SEH instead of DWARF based unwinding tables. The second is the "emulator compatible" abi, which as far as I understand it is basically the x86_64 abi except with x86_64 registers mapped onto AArch64 registers. This makes it easy to translate x86_64 to AArch64 with the "emulator compatible" abi and allow this translated code to be used in the same process as native AArch64 code with this abi in case you still have x86_64 dll's you need to use.

[Berrysoft]

The latter one is ARM64EC ABI, which seems to be not supported by Rust. I suggest focusing on the normal ARM64 ABI.

[kpreisser]

Hi, thanks for your replies!

Windows on AArch64 has two possible abi's. The first is I believe AAPCS compatible except with SEH instead of DWARF based unwinding tables. The second is the "emulator compatible" abi, which as far as I understand it is basically the x86_64 abi except with x86_64 registers mapped onto AArch64 registers.

👍 See also the Microsoft Documentation about the Windows ARM64 ABI convention: https://learn.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions

The latter one is ARM64EC ABI, which seems to be not supported by Rust. I suggest focusing on the normal ARM64 ABI.

Yes, ARM64 would also be the one we are interested in (.NET for example doesn't support ARM64EC).

There's also the question of testing in CI: we have Windows/x86-64 runners, but does Windows have a "reverse emulator" that can run aarch64 .exes on x86-64, like it can the other way around? If not, I suppose we could treat this the same as macOS/aarch64: support in-tree, but not tested so best-effort.

To my knowledge, Window ARM64 supports emulation for x86/64 processes, but unfortunately not the other way round. The only way I can think of to run Windows Arm64 binaries on x86-64 hardware would be using a VM with emulation, e.g. QEMU.

I would be happy to try to build wasmtime for Windows ARM64 and test it locally, to see if it basically works. Unfortunaltey I don't know Rust and am not very experienced with low-level handling. When I try to build using cargo build --target aarch64-pc-windows-msvc, I get the "unsupported platform" error as mentioned by @Berrysoft:

   Compiling wasmtime-environ v2.0.0 (C:\Users\Administrator\Desktop\Wasmtime-Test\wasmtime\crates\environ)
error: unsupported platform
  --> crates\runtime\src\traphandlers\windows.rs:62:17
   |
62 |                 compile_error!("unsupported platform");
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0425]: cannot find value `ip` in this scope
  --> crates\runtime\src\traphandlers\windows.rs:65:49
   |
65 |         let jmp_buf = info.take_jmp_buf_if_trap(ip, |handler| handler(exception_info));
   |                                                 ^^ not found in this scope

error[E0425]: cannot find value `ip` in this scope
  --> crates\runtime\src\traphandlers\windows.rs:71:31
   |
71 |             info.set_jit_trap(ip, fp);
   |                               ^^ not found in this scope

error[E0425]: cannot find value `fp` in this scope
  --> crates\runtime\src\traphandlers\windows.rs:71:35
   |
71 |             info.set_jit_trap(ip, fp);
   |                                   ^^ not found in this scope

https://github.com/bytecodealliance/wasmtime/blob/c1d6ca48a74ab6f356d08baa533b90b17ccb834a/crates/runtime/src/traphandlers/windows.rs#L54-L73

I think the (*(*exception_info).ContextRecord) refers to the arch-specific CONTEXT structure (here for x86_64), which for ARM64 is defined as alias for ARM64_NT_CONTEXT. However, I'm not sure which values/registers to use from that struct for the ip and fp variables, or if there are other changes required.

Edit: When I try to avoid that error by using the following code:

            } else {
                //compile_error!("unsupported platform");
                let ip = 0 as *const u8;
                let fp = 0;
            }

I get another build error due to the unsupported platform:

   Compiling wasmtime-jit v2.0.0 (C:\Users\Administrator\Desktop\Wasmtime-Test\wasmtime\crates\jit)
error: unsupported target platform for unwind
  --> crates\jit\src\unwind.rs:12:9
   |
12 |         compile_error!("unsupported target platform for unwind");
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0432]: unresolved import `crate::unwind::UnwindRegistration`
 --> crates\jit\src\code_memory.rs:3:5
  |
3 | use crate::unwind::UnwindRegistration;
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `UnwindRegistration` in `unwind`

error[E0282]: type annotations needed
  --> crates\jit\src\code_memory.rs:68:52
   |
68 |             unwind_registration: ManuallyDrop::new(None),
   |                                                    ^^^^ cannot infer type of the type parameter `T` declared on the enum `Option`
   |
help: consider specifying the generic argument
   |
68 |             unwind_registration: ManuallyDrop::new(None::<T>),
   |                                                        +++++

https://github.com/bytecodealliance/wasmtime/blob/c1d6ca48a74ab6f356d08baa533b90b17ccb834a/crates/jit/src/unwind.rs#L1-L14

Thanks!

[Berrysoft]

The value for ip and fp is here:

typedef struct _ARM64_NT_CONTEXT {
  // ...
  union {
    struct {
      // ...
      DWORD64 Fp;      // fp
      // ...
    } DUMMYSTRUCTNAME;
    // ...
  } DUMMYUNIONNAME;
  // ...
  DWORD64          Pc; // ip
  // ...
} ARM64_NT_CONTEXT, *PARM64_NT_CONTEXT;

[Berrysoft]

I'm not an expert to unwinding and don't have an ARM64 Windows device neither. But according to the docs I've seen recently, I think you can just try using the winx64 mod for ARM64. I'm not sure if it will work, and it's just some silly advice:)

Edit: with a slight modify to winx64 the wasmtime crate could be built targeting ARM64, but I cannot test it because I don't have ARM64 device.

@@ -21,7 +21,7 @@
         if RtlAddFunctionTable(
             unwind_info as *mut _,
             (unwind_len / unit_len) as u32,
-            base_address as u64,
+            base_address as _,
         ) == 0
         {
             bail!("failed to register function table");

[bjorn3]

I believe RtlAddFunctionTable is supposed to take a u64 argument as base address. What was the error you got without your change?

[Berrysoft]

To use it for ARM64. It uses usize on ARM64.

[kpreisser]

Thank you! With these changes I'm able to build wasmtime for aarch64-pc-windows-msvc successfully as debug, based on commit c1d6ca48a74ab6f356d08baa533b90b17ccb834a.

Testing the debug version on a Windows 11 Pro Version 22H2 (Build 22621) ARM64 machine, basic functionality seems to work, when I create a myModule.wast like this:

(module 
    (func (export "myTrap") 
        unreachable
    )
    (func (export "getVal") (result i32) 
        i32.const 12345
    )
)

and run it:

C:\Users\Name\Desktop\WasmTest>wasmtime myModule.wast --invoke getVal
warning: using `--invoke` with a function that returns values is experimental and may break in the future
12345

Unfortunately, trap handling doesn't seem to work. When I call wasmtime myModule.wast --invoke myTrap, wasmtime.exe crashes with the following error report in Windows Event Viewer:

Faulting application name: wasmtime.exe, version: 0.0.0.0, time stamp: 0x63372295
Faulting module name: ntdll.dll, version: 10.0.22621.232, time stamp: 0x0c898015
Exception code: 0xc00000ff
Fault offset: 0x0000000000129440
Faulting process id: 0x0xBAC
Faulting application start time: 0x0x1D8D4EFB3551177
Faulting application path: C:\Users\Name\Desktop\WasmTest\wasmtime.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 4b0bd570-7bc1-48ff-86a5-c28db648de5b
Faulting package full name: 
Faulting package-relative application ID: 

When attaching Visual Studio 2022 as debugger, I get:

Unhandled exception at 0x00007FFBD9119440 (ntdll.dll) in wasmtime.exe: 0xC00000FF: A malformed function table was encountered during an unwind operation.

The call stack is displayed as follows:

ntdll.dll!00007ffbd9119440()
ntdll.dll!00007ffbd9025810()
ntdll.dll!00007ffbd9096730()
vcruntime140.dll!00007ffbbbde1c5c()
vcruntime140.dll!00007ffbbbde1bb8()
wasmtime.exe!wasmtime_longjmp(void * JmpBuf) Line 61
	at C:\Users\VPC\Desktop\WebAssembly-Test\wasmtime\crates\runtime\src\helpers.c(61)
wasmtime.exe!wasmtime_runtime::traphandlers::windows::exception_handler::closure$0(wasmtime_runtime::traphandlers::windows::exception_handler::closure_env$0 info, enum$<core::option::Option<ref$<wasmtime_runtime::traphandlers::call_thread_state::CallThreadState>>, 1, 18446744073709551615, Some>) Line 75
	at C:\Users\VPC\Desktop\WebAssembly-Test\wasmtime\crates\runtime\src\traphandlers\windows.rs(75)
wasmtime.exe!wasmtime_runtime::traphandlers::tls::with<i32,wasmtime_runtime::traphandlers::windows::exception_handler::closure_env$0>(wasmtime_runtime::traphandlers::windows::exception_handler::closure_env$0 closure) Line 634
	at C:\Users\VPC\Desktop\WebAssembly-Test\wasmtime\crates\runtime\src\traphandlers.rs(634)
wasmtime.exe!wasmtime_runtime::traphandlers::windows::exception_handler(windows_sys::Windows::Win32::System::Diagnostics::Debug::EXCEPTION_POINTERS * exception_info) Line 49
	at C:\Users\VPC\Desktop\WebAssembly-Test\wasmtime\crates\runtime\src\traphandlers\windows.rs(49)
[External Code]

[cfallin]

That's progress! Thanks @kpreisser for trying that out!

The failure in trap handling is almost certainly to do with our lack of SEH (structured exception handling) metadata on aarch64; that will take some actual engineering work in the ABI code to do. It may not be too bad though, as we share many ABI details with x86-64 and most of the code is generic. I'm happy to point out the bits that are likely relevant if someone wants to work on it.

[jameysharp]

In my opinion that's enough progress to be worth a pull request: if it not only builds but actually runs correctly as long as nothing traps, that's a big improvement. If implementing SEH turns out to be hard, then I'd love to at least merge what you have now.

Also it occurs to me that if we could check in CI that Wasmtime at least builds for this target, that would give us some confidence we haven't broken it, even though it's not as good as running the full test suite. We should be able to cross-compile without having to figure out how to run Windows/aarch64 binaries. This might be a good project if anybody else is looking to contribute but doesn't feel comfortable fiddling with ABI details.

[cfallin]

A pull request to fix the compilation would be great, yeah -- let's get the above work in-tree!

I do want to be a little cautious about what we put in CI and build, though: "runs correctly as long as nothing traps" is a pretty precarious and non-standards-compliant thing and I wouldn't want, e.g., a bundle of build artifacts published where someone takes the .exe then wonders why we have a broken engine. I'm not sure if that's what you were suggesting or not, but just to say it explicitly, "passes core Wasm tests" would be my threshold for published artifacts.

I do think a cargo check for the platform wouldn't be a bad idea, though; that would prevent trivial regressions while this remains a pending task.

[jameysharp]

Yeah, I think cargo check does everything I actually wanted there. I guess there could be bugs that wouldn't be detected until link-time, but for anything else, cargo check should probably be as good as cargo build.

[Berrysoft]

I've opened PR to add this initial Windows ARM64 support. However, I don't know about the function table and the ARM64 SEH. I think we need some help from experts to improve the traps in the future.