wasi-sdk
wasi-sdk copied to clipboard
WebAssembly
WASI support
A few years ago, @binji ported LLVM/Clang 8 to WebAssembly/WASI: https://github.com/binji/wasm-clang (https://github.com/binji/llvm-project/releases/tag/wasm-bin-3)
This is extremely useful, as it allows bringing a modern compiler to many old systems, e.g. via wasm2c and https://github.com/turbolent/w2c2.
Ever since WASI SDK was released, I hoped that we could have a "self-hosting" build of it, i.e. WASI SDK compiled to WASI.
I "re-applied" Ben's changes on top of LLVM 15, the version that WASI SDK 19 uses. Unlike the original changes, I simply used the __wasi__ define, and did not change wasi libc / libcxx -- according to Ben, he faked threading support. Instead, I opted for also adjusting the code that relies on threading support and related data structures (mutex, conditional variables, etc.).
Diff: https://github.com/turbolent/llvm-project/compare/8dfdcc7b7bf66834a761bd8de445840ef68e4d1a...8e9c5068416bd9ca1144fa2fb659636b13bd4f60
As you can see, there were not too many changes needed to get it to compile, however, the changes are hacky at times. I hope that with the upcoming threading support of WASI / WASI SDK, it should be possible to re-enable much of the threading related code in LLVM.
I have not tested the resulting binaries of clang and wasm-ld too much, but they are able to compile a simple hello world program.
I'm opening this PR as a draft, as I would like to get some feedback and also this allows me to share the port with e.g. the llvmbox project.
For the threads-related code, you might need to use the wasm32-wasi-threads target and the wasi-sdk-20+threads pre-release.
If possible, could you share a link that shows the changes to upstream llvm that were required?
Doh! I saw you did that already right in the PR description, sorry.
I'm having trouble re-producing, but a thought occurred to me - could wasix-libc obviate the need to make changes to LLVM?
This is cool stuff!
For wasi-sdk here, we'll need the LLVM changes to be submitted upstream. At a quick glance, the changes look reasonable, though I do suggest changing #ifndef __wasi__ to something like #ifndef _REENTRANT, because as @abrown mentions above, WASI doesn't always mean no threads.
@sunny-g targeting WASIX instead of just WASI might reduce the changes needed to LLVM, but the result would only work in WASM runtimes with WASIX support -- AFAICS that is only wasmer at the moment
@sunny-g targeting WASIX instead of just WASI might reduce the changes needed to LLVM, but the result would only work in WASM runtimes with WASIX support -- AFAICS that is only wasmer at the moment
Most of the change look like they relate to threading... hopefully those will not be needed once we have some kind of "null" pthread implementation in wasi-libc.. the remaining changes should be upstream-able I believe.
I tried to update this to LLVM 18/HEAD :rocket: Edit: It turns out the mutex header needs to be fixed for it to be properly upstreamed to LLVM.
