Support for C++ exceptions

[alexcrichton]

I noticed that there wasn't already a tracking issue for C++ exceptions, so I wanted to open one up to at least track the progress of this. Currently wasi-sdk does not support C++ exceptions, and trying to fiddle with the build locally shows that it's not a turnkey solution at this time or a small PR to some build configuration. Some questions that I ran into locally are:

• Should exceptions support be provided in existing targets, or a new target? My assumption is that the entire C++ standard library needs to be built with/without exceptions meaning that technically -fexceptions wants an entirely different sysroot than -fno-exceptions. I don't believe that Clang currently dynamically selects a different sysroot based on this flag, so the next best thing we could do would be to add more targets. That would double the size of the target list, however.
• Right now using the standard wasm exceptions proposal requires -mllvm -wasm-use-legacy-eh=false, and I'm not sure if that would require tripling the sysroot where one uses legacy eh, one doesn't, and one uses no eh.
• Currently -fwasm-exceptions looks to be required, but I'm not sure if this is a temporary state of affairs or one that's intended to last.
• Wasi-sdk looks like it will need to build LLVM's libunwind. There's a few build issues with this where it doesn't natively build for wasm as-is (even with all exception-related things turned on). It's not clear to me if (a) libunwind is not expected to build for wasm, (b) libunwind should be buildable as-is but maybe with different configuration, or (c) perhaps there are Emscripten patches to libunwind which haven't made their way into LLVM itself.
• I don't know where the responsibility of passing -lunwind lies. Should the Clang driver automatically pass that? Should users be required to pass that?

These are the questions I've run into locally myself. From an implementation perspective all the "meat" of C++ exceptions is done in the sense that I was able to edit enough files to get something compiling and executing. The remaining bits of work are all on the toolchain side I believe (all of the above questions)

[dicej]

Regarding the existing targets vs. new targets question: perhaps a good first step would be to add support for building all existing targets with exceptions enabled, but not make that the default. So we'd continue publishing builds without exception support, but allow people to experiment with them by building from source.

[yamt]

Wasi-sdk looks like it will need to build LLVM's libunwind. There's a few build issues with this where it doesn't natively build for wasm as-is (even with all exception-related things turned on). It's not clear to me if (a) libunwind is not expected to build for wasm, (b) libunwind should be buildable as-is but maybe with different configuration, or (c) perhaps there are Emscripten patches to libunwind which haven't made their way into LLVM itself.

there are at least a few attempts to fix the build.

https://gist.github.com/yamt/b699eb2604d2598810a2876ff2ffc8d8 (mine) https://github.com/llvm/llvm-project/pull/140365 https://github.com/llvm/llvm-project/pull/156087 (i haven't looked at this one closely)

[whitequark]

See also https://github.com/WebAssembly/wasi-sdk/issues/334, which is less detailed than this issue but was filed earlier.

[yerzham]

I applied changes to libunwind from https://gist.github.com/yamt/b699eb2604d2598810a2876ff2ffc8d8 and added libunwind to be included in wasi-sdk sysroot build. I made sure to use -fwasm-exceptions and -mllvm -wasm-use-legacy-eh=false where possible, but I kept having issues with __cpp_exception being undefined when building sysroot.

Some debugging revealed that code around https://github.com/llvm/llvm-project/pull/159143 used by emscripten toolchain was likely responsible for that. Defining __cpp_exception tag moved to the emscripten toolchain. Previous version wouldn't have worked anyway because fPIC is enabled for wasip2. Correct me if I am wrong.

So I kept the __cpp_exception tag emission and marking it weak (whatever weak means, idk) if it is targeting WASI instead of checking if isPositionIndependent. I suppose defining __cpp_exception can be also moved to wasi-sdk/libc just like emscripten did?

At the end, I was able to build the whole wasi-sdk with patched llvm project, and successfully compiled a C++ code with exception throws. In my set up I did not have to pass -fwasm-exceptions or -lunwind during the sample C++ code compilation. Running it with wasmtime run -W exceptions=y worked. Without the exception feature, wasmtime correctly errored with exception refs not supported without the exception handling feature.

I hope my couple day debugging session around this helps :)

EDIT: relevant conversation already happened at https://github.com/llvm/llvm-project/issues/103592#issuecomment-3299970552

[yerzham]

Ok, my attempts to fix the older version of llvm project were irrelevant, sorry. As a note for people like me who used the latest release of wasi-sdk (28) and wanted to understand if and when C++ exceptions are going to be supported:

1. The changes from https://github.com/llvm/llvm-project/pull/159143 and https://github.com/llvm/llvm-project/pull/160959 already address the support for compiling code with C++ exceptions to Wasm, moving emscripten related code out in favor of standard exception handling. With this, compiler-rt needs to be built with -fwasm-exceptions too.
2. Now this issue is tracking how to enable the C++ exception handling in wasi-sdk.

[whitequark]

I think the biggest difficulty here is that it makes sense to ship both exceptions-enabled and non-exceptions-enabled builds of the sysroot side by side, but building this and threading it through the compiler is not quite trivial.

[torokati44]

it makes sense to ship both exceptions-enabled and non-exceptions-enabled builds

What's the argument for exceptionless builds? I suppose, technical details like "smaller", "faster", etc.? AFAIK working exceptions are a non-optional part of C++, therefore, if a toolchain can't do exceptions, it can't really do C++.…? More like "C with classes"? 😛

[whitequark]

There are runtimes in active widespread use that do not have exceptions and may not gain support for exceptions quickly. As much as I wish exceptions were available 5 years ago (seriously, I've had to patch so much stuff to build on, essentially, non-conformant C++ compilers) the reality is that they still aren't uniformly available. (For example, Wazero still does not implement EH.)

That being said, if wasmtime fully supports exceptions now, I personally would not shed a tear for exception-less builds of WASI SDK. (It used not to not so long ago, at least...)

[whitequark]

Looks like the status as of Wasmtime 37 is:

Wasmtime now fully implements the WebAssembly exception-handling proposal. Support is still disabled by default but is ready for testing. The proposal will be enabled by default in a future release of Wasmtime. https://github.com/bytecodealliance/wasmtime/pull/11326

[torokati44]

runtimes in active widespread use that do not have exceptions

Sounds like chicken and egg issue?

[whitequark]

I think it is mostly a result of implementation complexity, coupled with the fact that the EH spec only recently became fully usable to implementers.

[yerzham]

I tested wasi-sdk build result with the newest version wasmtime and it worked, yes. I was super happy to see C++ with exceptions working without requiring me to rewrite a big-ish library to work without exceptions.

As starters, I would love to see some experimental support added for building wasi-sdk from source with exceptions enabled with short docs included. I can make a PR, but I do not trust myself on this. Also, libunwind needs to be patched first anyway. That would make it easier for people experimenting with large C++ codebases to get into WASI without too much pain in between.

As for the question of adding additional sysroot builds and how to distribute the builds, that can be done later?

[whitequark]

I can review the libunwind LLVM PR if you make one.

[alexcrichton]

Personally I think the best path forward would be to first get everything landed/implemented such that locally wasi-sdk can be built with/without exceptions (e.g. a CMake-based toggle). At that point I think it would be worth taking stock again to figure out how exactly to manage its rollout. For example we'd, at that point, probably be better equipped to answer the questions of:

• Should this be a new compiler target?
• Should this have a separate sysroot which is selected on the basis of compiler flags?
• Should exceptions get enabled-by-default in wasi-sdk?

Figuring out how best to roll this out is going to require a balance of concerns with what's possible in the ecosystem engine-wise and what the cost of various options here are in terms of toolchain complexity. The easiest option here is likely to be "just turns exceptions on by default" in wasi-sdk, but that would require modules to be runnable in a breadth of runtimes because turning off exceptions won't be easy (it'd require using an older wasi-sdk or compiling your own wasi-sdk). The hardest option is probably "ship multiple sysroots and select the right one based on compiler exception-related flags". I'm not even sure if that would work but it's at least worth writing down as an option I think.

[sbc100]

• Should this be a new compiler target?
• Should this have a separate sysroot which is selected on the basis of compiler flags?
• Should exceptions get enabled-by-default in wasi-sdk?

Another option is to use new subdirectory within the existing sysroot. This is what we do for the LTO variants: https://github.com/llvm/llvm-project/blob/6f5c8fe1c1d24604d3328b82f5a1ed348e59326f/clang/lib/Driver/ToolChains/WebAssembly.cpp#L259-L265

This way the we would only need to compile/ship the system libraries that actually use exception in this mode (i.e. libc++ and libc++abi, and not libc or compiler-rt).

[yerzham]

If this helps anyone locally experimenting with exceptions-enabled builds of wasi-sdk sysroot: https://gist.github.com/yerzham/302efcec6a2e82c1e8de4aed576ea29d.

Works after https://github.com/llvm/llvm-project/pull/168449 lands here.

Tested with different C++ exception scenarios using wasmtime 38, basically has same limitations as emscripten.

[yerzham]

(c) perhaps there are Emscripten patches to libunwind which haven't made their way into LLVM itself.

I see libunwind in emscripten is built only with Unwind-wasm.c and libunwind/include header files, and not full LLVM runtimes build? This explains why it works fine under emscripten.

https://github.com/emscripten-core/emscripten/blob/55aef9811e0bb3ed49f58691090f810046a40f22/tools/system_libs.py#L1724-L1736

[hmelder]

I'm currently porting Objective-C to WebAssembly including EH. https://github.com/llvm/llvm-project/pull/169043 tracks a bunch of assumptions/hacks that the Emscripten people used in LLVM. It might be of interest for you as well.