mlton icon indicating copy to clipboard operation
mlton copied to clipboard

Add WASI (WebAssembly) cross compile support

Open agoode opened this issue 1 year ago • 5 comments

Fixes #549.

See https://github.com/agoode/mlton/blob/wascally/doc/guide/src/RunningOnWASI.adoc for instructions.

I've only tested with macOS, but I could try Linux also.

This makes a bunch of improvements to cross compilation support in general, since WASI is really always going to be a cross target.

Potentially controversial is leaving unimplemented POSIX functions missing from wasi.c. I think it works well to have this become a linker error instead of a runtime error, but I am open to suggestions.

Thanks to @jcreedcmu for tricking me into doing this.

agoode avatar Feb 13 '24 03:02 agoode

I've also successfully tested this PR on linux. For what it's worth, here is the detailed script of what I ran to build (although I think @agoode 's existing documentation in this PR suffices)

Additionally, I was able to build a fairly nontrivial project, namely the twelf logical framework, and it executed fine on a WASI runtime. A demo is available at https://jcreedcmu.github.io/twelf-wasm/ --- this was exactly the thing that I hoped this PR would enable. This exercises at least file IO as a proof of concept.

######### wasi-sdk
cd $BUILD
git clone --recursive https://github.com/WebAssembly/wasi-sdk.git
cd wasi-sdk
sudo apt install cmake
sudo apt install clang
sudo apt install ninja-build
NINJA_FLAGS=-v make package

######### gmp
cd $BUILD
export WASISDK=$BUILD/wasi-sdk/build/install/opt/wasi-sdk
wget https://gmplib.org/download/gmp/gmp-6.3.0.tar.xz
tar xvf gmp-6.3.0.tar.xz
cd gmp-6.3.0
./configure --host=wasm32-unknown-wasi \
            CC=$WASISDK/bin/clang \
            RANLIB=$WASISDK/bin/ranlib \
            CFLAGS=-D_WASI_EMULATED_SIGNAL LDFLAGS=-lwasi-emulated-signal \
            --prefix=$BUILD/gmp-wasi-INSTALL
make

######### wasmtime
curl https://wasmtime.dev/install.sh -sSf | bash

######### mlton binary
cd $BUILD
git clone https://github.com/agoode/mlton agoode-mlton
cd agoode-mlton
export MLTON=$BUILD/mlton-INSTALL
make all
make PREFIX=$MLTON install

######### mlton wasm runtime
make clean
make CC=$WASISDK/bin/clang \
     AR=$WASISDK/bin/ar \
     RANLIB=$WASISDK/bin/ranlib \
     TARGET_OS=wasi \
     TARGET_ARCH=wasm32 \
     TARGET=wasm32-unknown-wasi \
     WITH_GMP_DIR=$BUILD/gmp-wasi-INSTALL \
     PREFIX=$MLTON \
     dirs runtime install-runtime

######### twelf
cd $BUILD/twelf
git checkout jcreed/mlton-wasm
$MLTON/bin/mlton -target wasm32-unknown-wasi build/twelf-server-mlton-wasm.mlb

######### run twelf
wasmtime ./build/twelf-server-mlton-wasm
# Twelf 1.7.1+ (r, built 02/03/24 at 15:21:14 on system76-pc)
# %% OK %%

jcreedcmu avatar Feb 13 '24 18:02 jcreedcmu

This looks very nice; I'm pleasantly surprised at how minimal the PR is to support WebAssembly/WASI.

MatthewFluet avatar Feb 15 '24 00:02 MatthewFluet

There are some improvements coming to the wasi-sdk (notably https://github.com/WebAssembly/wasi-sdk/pull/388), so I would maybe avoid merging this PR right now in favor of a more simple one that may come soon.

agoode avatar Mar 04 '24 16:03 agoode

@agoode is this PR ready to be merged? It seems that it is working well for the Twelf project.

MatthewFluet avatar May 13 '24 14:05 MatthewFluet

I'm going to split it up into a few smaller PRs. There are some good changes that can go in now, and a few others I want to revisit for the updated wasi-sdk-22. I'll send some new PRs when they are ready.

agoode avatar May 13 '24 14:05 agoode

Looks good. (Either I missed it or changing to "ready for review" didn't re-raise the issue in my notifications.) In runtime/Makefile, I will probably change the WASMTIME variable to something less WASM-specific, since it could be useful for other wrappers that allow one to emulate a target system on the build system.

MatthewFluet avatar May 21 '24 14:05 MatthewFluet