Compilation errors and functional correctness vary based on WASI-SDK Optimization levels.

[bushidocodes]

This bug is related to WASI-SDK work and thus is in the branch wasi-abstract-interface.

Currently, at least two tests trigger a fatal LLVM error during the final clang step linking the *.bc emitted from aWsm with the uvwasi runtime:

error: Invalid instruction with no BB (Producer: 'LLVM11.1.0' Reader: 'LLVM 11.1.0')

This suggests that perhaps the LLVM bitcode emitted by the compiler is invalid

Additionally, one test is functionally correct at optimization level, but returns a different result at a different optimization level. This suggests that different WebAssembly opcodes emitted at different optimization levels may cause a control flow bug.

Here follows steps to recreate the bugs and a zip containing the *.wasm files of the two apps at optimization levels 0-3 with associated textfiles containing opcode counts of each variant.

Steps to Replicate

1. Checkout wasi-abstract-interface. The setup scripts likely differ from what you already have, so you may need to run the setup script.
2. Navigate to awsm/test/wasi

The first test I want you to look at is in awsm/test/wasi/c/atof.c This program deserializes arg 1 into a float and then serialized it back out to stdout.

#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char** argv) {
    double pi = atof(argv[1]);
    printf("%.2lf\n", pi);
}

You can build and run this natively with the following to get a feel for it:

sean@MANPUTER:~/projects/awsm/tests/wasi$ clang c/atof.c
sean@MANPUTER:~/projects/awsm/tests/wasi$ ./a.out 3.14
3.14
sean@MANPUTER:~/projects/awsm/tests/wasi$ rm a.out

It turns out that the behavior of this program varies on the optimization level passed to the WASI-SDK clang compiler.

At optimization level 3,

wasm/%.wasm: c/%.c
	@mkdir -p wasm
	${WASMCC} ${WASMLINKFLAGS} ${OPTFLAGS} -O3 $< -o $@

everything builds and runs correctly:

sean@MANPUTER:~/projects/awsm/tests/wasi$ ./vm/atof_vm 3.14
3.14

At optimization levels 1 and 2:

wasm/%.wasm: c/%.c
	@mkdir -p wasm
	${WASMCC} ${WASMLINKFLAGS} ${OPTFLAGS} -O2 $< -o $@

aWsm compiles correctly, but the final clang linking with the runtime fails with an error error: Invalid instruction with no BB (Producer: 'LLVM11.1.0' Reader: 'LLVM 11.1.0') that suggests that maybe LLVM bitcode emitted by the compiler is invalid:

sean@MANPUTER:~/projects/awsm/tests/wasi$ make clean vm/atof_vm
rm -f ./wasm/* ./bc/* ./vm/*_vm
/opt/wasi-sdk//bin/clang --sysroot=/opt/wasi-sdk//share/wasi-sysroot -Wl,--allow-undefined,-z,stack-size=32768,--threads=1 -O3 -flto -O2 c/atof.c -o wasm/atof.wasm
../../target/release/awsm  wasm/atof.wasm -o bc/atof.bc
INFO [awsm] running awsm("wasm/atof.wasm", Some("bc/atof.bc"))
INFO [awsm::wasm] Globals taking up 4 bytes
INFO [awsm::wasm] Data initializers taking up 2239 bytes
INFO [awsm::wasm] Function table entries 4 (ignoring fragmentation)
INFO [awsm::codegen::memory] memory limits ResizableLimits { initial: 1, maximum: None }
INFO [awsm] awsm finished successfully
clang -pthread -ldl -lm -O3 -flto  -I../../runtime/libc/wasi/include -I../../runtime/thirdparty/dist/include bc/atof.bc ../../runtime/runtime.c ../../runtime/libc/wasi/wasi_main.c ../../runtime/libc/wasi/wasi_backing.c ../../runtime/libc/wasi/wasi_impl_uvwasi.c ../../runtime/libc/env.c ../../runtime/memory/64bit_nix.c ../../runtime/thirdparty/dist/lib/libuvwasi_a.a ../../runtime/thirdparty/dist/lib/libuv_a.a -o vm/atof_vm
error: Invalid instruction with no BB (Producer: 'LLVM11.1.0' Reader: 'LLVM 11.1.0')
1 error generated.
make: *** [Makefile:24: vm/atof_vm] Error 1
rm bc/atof.bc wasm/atof.wasm

At optimization level 0:

wasm/%.wasm: c/%.c
	@mkdir -p wasm
	${WASMCC} ${WASMLINKFLAGS} ${OPTFLAGS} -O0 $< -o $@

Everything compiles correctly, but the program does not work as expected:

sean@MANPUTER:~/projects/awsm/tests/wasi$ ./vm/atof_vm 3.14
0.00

The Fibonacci app works a bit differently. It is only able to compile at optimization level 0

wasm/%.wasm: c/%.c
	@mkdir -p wasm
	${WASMCC} ${WASMLINKFLAGS} ${OPTFLAGS} -O0 $< -o $@

sean@MANPUTER:~/projects/awsm/tests/wasi$ echo "10" | ./vm/fibonacci_vm 
55

At optimization levels 1, 2, and 3, it errors out in the final linking step with the same error message as above error: Invalid instruction with no BB (Producer: 'LLVM11.1.0' Reader: 'LLVM 11.1.0')

Here is the zip containing the wasm files and associated opcode counts. Despite the issues identified above, all of these *.wasm files correctly execute on wamtime: sample.wasm.zip