Yellow-Dog-Man
WebAssembly support for general dynamic scripting & custom functionality
Is your feature request related to a problem? Please describe.
Currently the only way to create dynamic content in Resonite is using the ProtoFlux scripting language. This, while powerful, can be in some cases limiting, particularly when creating complex programs, but also from the perspective of tooling, creating programs (which has to be done visually instead of typing), porting existing code and experience from other languages.
Describe the solution you'd like
To provide generalized support for scripting, we plan to integrate the WebAssembly (WASM for short) standard into Resonite. This will involve providing an interface/API for the WASM modules to create new components in-game, with specializations for various other types of components:
- Procedural assets (textures, meshes, audio...)
- ProtoFlux nodes (meaning you'll be able to define new nodes with their behavior implemented in WASM)
- Import/Export modules (to extend the supported formats)
- Device drivers (for adding support for more device types)
WASM has following major advantages for Resonite:
- Wide tooling & language support - this will allow variety of languages to be used to code for Resonite - C/C++, Rust, Go, Java, C#, PHP, COBOL (if you really want) and through some of these also introduce support for other languages (by compiling their runtimes into WASM), such as Lua or Python - see more here: https://www.fermyon.com/wasm-languages/webassembly-language-support
- It's sandboxed & designed to be secure - this will allow WASM modules to be synchronized in the session and loaded dynamically, unlike plugins/mods which require manual installation. After switch to process based architecture (https://github.com/Yellow-Dog-Man/Resonite-Issues/issues/706) we might also be able to add additional process-level sandboxing, which will provide even stronger protection (double layer sandbox)
- Support for existing code - given the large language support, a lot of existing libraries can be brought in, rather than needing to be re-implemented - texture/audio/mesh processing libraries, emulators and so on
- It's designed for long term compatibility - if we keep the ABI/API stable, the modules will keep functioning for a long time and preserve user-made content
We will also likely provide tooling for some popular languages (e.g. C/C++ and Rust) to get things started, but will leave others left to the community contributions.
Additionally, one particular challenge will be a system for resolving dependencies for WASM modules - e.g. when multiple WASM modules reference a common shared library. This will be provided by additional/separate system (I will make issue on this also).
Major part of our work will be defining the ABI/API for interfacing with the FrooxEngine, which will be important to ensure the long term stability and compatibility of the modules.
At the time of writing this, the runtime we plan to use is Wasmtime, given it's good C# support, very active development and support from large organizations.
Describe alternatives you've considered
We were considering integrating a specific scripting language, such as Lua or Python, however this would be much more limited and would require users to use that specific language.
With WASM, those languages can still be used, but the support is not limited to just those specific languages.
Additional Context
There was existing issue created on this. However given the large impact of this feature and it's scope, I've made this official issue, since the goal is to focus on high-level benefits and support, rather than very low level specifics (which will come when this is actually prioritized): https://github.com/Yellow-Dog-Man/Resonite-Issues/issues/573
This would've been great when I was working on Handless Helper, as ProtoFlux lacks an elegant analog for Functions ans Function Calls.
As someone who has taught numerous programming languages, including JavaScript, this would definitely be something I would use to build some reusable libraries and complicated systems, and I know I'm not the only one with programming experience.
Functions and libraries will come to ProtoFlux first in form of nested nodes.
question, with WASM possibly coming to resonite. would it be seperate from protoflux or would it require Protoflux to use?
because my issue with WASM at this moment is that protoflux might become obsolete or at least outdated with this and having some tie into it would be nice to keep the both here.
It will be separate, but integrated. You will not need ProtoFlux to use WASM. But you'll be able to use WASM to extend and interface with ProtoFlux.
I do recommend reading the original post, because it covers some of it.
Just spitballin' here, Found hyperlight a while back and I thought it could be used as a possible starting point.
https://github.com/hyperlight-dev/hyperlight https://github.com/hyperlight-dev/hyperlight-wasm
"specifically optimized for securely running untrusted code with minimal impact" which would be ideal for Resonite, also support Windows and Linux, but there aren't any C# bindings.
@codecnotsupported I don't quite understand how does that relate to Web Assembly? WASM already provides its own secure VM.
@codecnotsupported I don't quite understand how does that relate to Web Assembly? WASM already provides its own secure VM.
Hyperlight is a runtime for running unsecure code akin to wasmtime, but Hyperlight is not only for WASM. Hyperlight-wasm is a WASM runtime that builds on top of hyperlight. I wanted to propose it as a alternative to Wasmtime, but in retrospect Wasmtime is probably better suited for your use case.
I see! Thank you for the additional info!
Yeah I do think Wasmtime is probably better. There's a few factors for us we use:
- It's Bytecode Alliance supported project, meaning a lot of momentum behind it
- It has generally more support and popularity, which gives us more confidence in relying on it
- It has well maintained C# bindings
We can look at alternatives, but generally these factors will be hard to go against.
Would WASM integration allow us to get into the Finer details of FrooxEngine. say for things like changing how certain parts of the engine react to user interaction or creating reactive world settings that change based on world host's local pc time like the world name?
It depends on specifics. You'll be able to interface with the data model and some engine parts. You won't be able to change literally everything, but it'll give you a lot of control and power.
Is it possible that when this is implemented we could have a tool that is a notepad with a flippable single page (and the opinion of a two-page notebook)?
Also, is this feature actively being worked on? A major reason I (and a friend of mine) have held off on using Resonite these days is the lack of this feature, along with issue #1233 not being implemented yet.
@RabbitRetro That would be completely separate thing from this feature. You could implement something like that yourself too.
This is not being actively worked on right now. You can see that on the right panel where it's "Todo":
If it was actively worked on, it would say "In progress"
This is not being actively worked on right now. You can see that on the right panel where it's "Todo":
Venturing into offtopic but I'm not sure the "Issue Triage & Tracking" project's information is available for us to see.
It doesn't appear in either of the places I would expect it to be if it was.
