Failure to execute AOT code compiled to 32bit --target=i386

[iKlask]

trafficstars

I've been using WAMR with intentions of embedding it within a 32bit app on windows. After pulling 2.0.0 I've noticed my AOT execution gets stuck in iwasm. Specifically inside the invoke_native call. If I compile for x86_64 AOT it works fine in x64 iwasm.

I've attached a simple wasm example (Fibonacci function). Both the source .wasm file and the compiled .aot from wamrc on 2.0.0 for --target=i386 wasm.zip

[wenyongh]

@iKlask 32bit may take longer time to run fib with the same argument, and it may take very long time to run fib with large input argument, had you tested the 32bit AOT file with a small argument? e.g. iwasm -f fib fib_i386.aot 10?

[iKlask]

yes I tried with many different arguments. My usual for testing was 10, 25, 30, and 40. This did work at one point since I had a benchmark setup. Hadn't tried AOT on 32bit in a while and now it locks up. I don't know if its an issue with whats generated in WAMRC, or if its an issue with the runtime.

[wenyongh]

Hi, maybe you can dump some logs to check whether the AOT file gets stuck, here is the wast file I refactored based on you wasm file, and note that I added some lines at the end:

(module
  (type $0 (func (param i32) (result i32)))
  (import "spectest" "print_i32" (func $print_i32 (param i32)))
  (export "fib" (func $1))

  (func $1 (type $0)
    (param $0 i32)
    (result i32)
    (local $1 i32)
    local.get $0
    i32.const 2
    i32.lt_u
    if $if
      local.get $0
      return
    end ;; $if
    local.get $0
    i32.const 2
    i32.sub
    call $1
    local.get $0
    i32.const 1
    i32.sub
    call $1
    i32.add

    ;; the below lines are added to dump logs
    local.tee $1
    local.get $0
    i32.const 30     ;; dump log if $0 >= 30, you can change it if needed
    i32.ge_u
    if
      local.get $0
      call $print_i32
      local.get $1
      call $print_i32
    end
  )
)

And then compile it into the wasm file, e.g. wat2wasm -o test.wasm test.wast, and compile it into AOT file. And when build wamr runtime, add -DWAMR_BUILD_SPEC_TEST=1 for cmake. The logs like below can be dumped:

in specttest.print_i32(30)
in specttest.print_i32(832040)
in specttest.print_i32(30)
in specttest.print_i32(832040)
in specttest.print_i32(31)
in specttest.print_i32(1346269)
in specttest.print_i32(32)
in specttest.print_i32(2178309)

You can modify the const value in line i32.const 30 to adjust the threshold. If it keeps dumping logs, I think it should not get stuck, otherwise it gets stuck.

[iKlask]

I reduced the threashold multiple times and it never logs. I even went down to 1 and nothing. Here is an example of fib(5) with a threshold of 1 on both x86 and x64 iwasm/AOT. The x86 version hangs before it can even log. the x64 version works fine

iwasm-x86.exe -v=5 -f fib fibLog.aot 5
[19:59:55:568 - D03CC8]: Load module success.

[19:59:55:569 - D03CC8]: Memory instantiate:
[19:59:55:569 - D03CC8]:   page bytes: 16384, init pages: 1, max pages: 1
[19:59:55:569 - D03CC8]:   data offset: 0, stack size: 0
[19:59:55:569 - D03CC8]:   heap offset: 0, heap size: 16384

^C
iwasm-x64.exe -v=5 -f fib fibLogX64.aot 5
[20:00:01:327 - 7FF6F4F14750]: Load module success.

[20:00:01:327 - 7FF6F4F14750]: Memory instantiate:
[20:00:01:327 - 7FF6F4F14750]:   page bytes: 16384, init pages: 1, max pages: 1
[20:00:01:327 - 7FF6F4F14750]:   data offset: 0, stack size: 0
[20:00:01:327 - 7FF6F4F14750]:   heap offset: 0, heap size: 16384

in specttest.print_i32(2)
in specttest.print_i32(1)
in specttest.print_i32(3)
in specttest.print_i32(2)
in specttest.print_i32(2)
in specttest.print_i32(1)
in specttest.print_i32(2)
in specttest.print_i32(1)
in specttest.print_i32(3)
in specttest.print_i32(2)
in specttest.print_i32(4)
in specttest.print_i32(3)
in specttest.print_i32(5)
in specttest.print_i32(5)
0x5:i32

[wenyongh]

How do you create the iwasm.sln and related make files for x86 on windows? I use cmake .. -DWAMR_BUILD_TARGET=X86_32, but the project configuration of generated iwasm.sln is set to x64 by default, it is not expected.

[iKlask]

Yes, I do use -DWAMR_BUILD_TARGET=X86_32 for my 32bit builds of iwasm

[wenyongh]

Hi, please try applying below patch, rebuild wamrc and then use it to generate aot file again:

diff --git a/core/iwasm/compilation/aot_llvm.c b/core/iwasm/compilation/aot_llvm.c
index df07c3ca..e1a13433 100644
--- a/core/iwasm/compilation/aot_llvm.c
+++ b/core/iwasm/compilation/aot_llvm.c
@@ -689,10 +689,12 @@ aot_add_llvm_func(AOTCompContext *comp_ctx, LLVMModuleRef module,
     bh_assert(LLVMGetReturnType(func_type) == ret_type);

     const char *prefix = AOT_FUNC_PREFIX;
-    const bool need_precheck =
+    bool need_precheck =
         comp_ctx->enable_stack_bound_check || comp_ctx->enable_stack_estimation;
     LLVMValueRef precheck_func = NULL;

+    need_precheck = false;
+
     if (need_precheck) {
         precheck_func = aot_add_llvm_func1(comp_ctx, module, func_index,
                                            aot_func_type->param_count,

This disables the native stack check boundary check in the generated the AOT file, but I am not sure why it causes the failure, it requires more effort to look into it.

[iKlask]

okay that patch does seem to work for 32bit AOT

[iKlask]

I'm not sure what has changed but I haven't been able to get win32 AOT working on any 2.0.0+ update. in iwasm when invoke_native is called it seems to jump to incorrect or bad instructions. Usually it instantly returns.

I've tried both on the releases for 2.0.0, 2.1.0, and 2.2.0. My wamrc command to compile for 32 bit windows has been very simple: wamrc --target=i386 -o output.aot input.wasm

I have tried both with and without this need_precheck = false; and it doesnt seem to change how my iwasm executes the aot code.