Support multiple WebAssembly engines

[autodidaddict]

In the current implementation, wasmex provides an elegant Rust/NIF wrapper around the wasmer engine. One of the compelling stories of WebAssembly is that, in the right circumstances, people can choose which engine they like so that they can tune the behavior of the runtime to specific needs. For example, you might prefer interpreted over JITted when running in constrained environments, you might choose a micro-targeting runtime versus a large, robust, "cloudy" runtime.

As we've discussed in this issue in wasmCloud, it would be amazing if we had some kind of meta-package which then let wasmex consumers (such as wasmCloud) choose which of the underlying engines would be used. Ideally, this would also allow wasmCloud to release with specific optimizations for cloud, IoT, edge, etc.

Per suggestion in the original issue, I've created an issue here so that we can discuss/plan/design the work to bring this about.

[tpmccallum]

Hi Kevin

Would it be reasonable to suggest that wasmex could provide wasmex consumers the choice of execution engine via Rust features?

For example, add each of the available engines to the Cargo.toml as a separate feature.

Then each engine could be implemented in the code i.e. allow wasmex to execute_function using WasmEdge if the WasmEdge feature is enabled.

The process for each engine is very similar i.e. read the .wasm file, instantiate the VM, run the function. There might be the odd nuance i.e. WasmEdge would have an additional function to create an AOT optimized executable.

This WasmEdge quickstart in Rust shows the similarities in syntax < https://github.com/WasmEdge/WasmEdge/blob/master/bindings/rust/wasmedge-sdk/examples/quickstart.rs >

Would love to hear your views on using Rust features for this parallel integration.

Thanks

[juntao]

What if multiple features are selected at the same time? Would there be an API call (eg config.defaultVM) to allow the user to choose a VM at runtime? Thanks.

[tpmccallum]

We could allow the compiler to generate an error when two incompatible features are simultaneously enabled.

#[cfg(all(feature = "wasm_engine_1", feature = "wasm_engine_2"))]
compile_error!("feature \"wasm_engine_1\" and feature \"wasm_engine_2\" cannot be enabled at the same time");

However, I guess this is a potential overhead for developers; so perhaps one of the following options might be better:

• use a config file in the same package
• create separate packages for each engine
• use environment variable or CLI variable (if/where possible)

I am guessing that system and CLI variables do not suit this arrangement and that separate packages would be too disparate. Therefore, perhaps a config file might be the simplest and most robust method.

[juntao]

I like both the cargo feature and config file options.

@autodidaddict Kevin, what do you think?

[autodidaddict]

How would a consumer of wasmex (eg wasmCloud) select the feature?

[juntao]

With the "features" option, the user need to build wasmCloud from source. I guess that might not be optimal since wasmCloud's preferred distribution method is binary installation packages.

The config file approach will allow users to select runtimes themselves but at an added cost of a slightly larger install package.

[autodidaddict]

This could possibly be my lack of experience here, but wasmex is Elixir, and wasmer and wasm edge are Rust, and wasmCloud is Elixir - and so I'm confused.

What I'm trying to figure out is how does the wasmCloud source (elixir) tell the wasmex project (Elixir+Hex) which underlying engine to use? Or does the wasmCloud source let people optionally choose things like wasmex-wasmedge and wasmex-wasmer ?

[tessi]

I'm really late to the party (sorry!) but what if wasmex becomes an interface + tooling package that then has sub-dependencies (wasmex-wasmer, wasmex-wasmedge, ..) for actual execution? These sub-dependencies would do the rust-interfacing (or utilize whatever is needed to execute wasm).

This way, the actual engine could be a config option of the wasmex genserver. We could even have two GenServers running, each using a different engine. What do you think?

[autodidaddict]

That sounds ideal, actually!

[tessi]

cool, so let me think some shower thoughts out loud. I would be very interested in your thoughts too (but see that you might not be too much into wasmex internals to have strong opinions).

a) wasmex (the "umbrella package") would define a common API against all sub-packages (wasmex-wasmer, wasmex-wasmedge, wasmex-wasmtime, ...). Finding a good API that serves all engines well is probably a hard problem. It also means that all engines must be based on the same abstractions (modules, memory, wasi file mappings) or we won't find a common API. Especially since I really only know wasmer right now, I might be blind to potential pitfalls. But this is probably something we can iterate on and don't need to get perfect on first try.

b) So let's assume we start by splitting wasmex from wasmex-wasmer and ignore the other engines for now. Where would we even split the package best? I know we need a GenServer to handle async messages and to allow calls from rust back into elixir.

1. That GenServer could live at wasmex (the umbrella) if we assume all engines will use that GenServer strategy and will use the same API between the GenServer and engine-implementation. It probably saves us some heavy lifting (the rust-calling-into-elixir trick isn't trivial). On the other hand, it means tight coupling between wasmex and its engines.
2. That GenServer could also live at the engine (wasmex-wasmer) so each engine must do the heavy lifting on its own. This would mean that wasmer, the main package, would only be a thin wrapper - maybe only a behaviour, docs, integration tests, and some tooling. It means more freedom for each engine to implement their own strategy. But also likely means some code duplication between engines (e.g. they all need to implement the weird rust-calling-into-elixir trick). I'm not sure I want to maintain multiple engines if each engine is a complicated snowflake.

My gut feeling says, 2. might be the better strategy because it leads to less tight coupling and allows us to experiment more.

c) I don't know if/when I personally have the time to implement these ideas. Offline life currently needs most of my attention.

[tessi]

As discussed here I am in the process of rewriting wasmex to wasmtime as an exercise to see how different the different WASM engines can be and where wasmex needs to be flexible.

My wasmtime fork is not fully working yet, but I already see that there are a lot of differences between just these two engines alone. (Although it seems that wasmer 3.0 will be getting closer to what wasmtime is doing, forcing me to refactor wasmex anyways). My gut feeling is that I can't maintain meta-wasmex plus all the engine-specific wasmex'es in my freetime. So, I guess supporting many WASM engines is not a way I can go with wasmex unfortunately.

Instead, I consider rewriting wasmex to wasmtime. Wasmtime seems to be closer to new development in the WASM space and I personally feel better depending on wasmtime as it's more of a community project and not based on a VC funded company.

What do you all think?

[tessi]

Closing this in favour of #336 Thanks for all the input, everyone! ❤️