WASM support

[tversteeg]

Would it fit the project to add a WebAssembly target?

• Creating a simple HTML5 canvas object and then copying the buffer of pixels every frame to JavaScript and sending the events back. Or it could be done using ImageData.
• Scaling could be implemented like this.
• A example for keyboard input is shown here.
• The menus could be a problem but that could be fixed with custom buttons in JavaScript.

I would love to help with developing/testing this if it fits your vision of the project!

[emoon]

That is an interesting question. I haven't actually thought about that. I have very little experience with web and esp WebAssembly but if you would want to implement this version I would be very happy with including it in the master branch.

As for menus Linux doesn't also support menus because there is no native API for doing them so I think that is fine.

[FloVanGH]

I've created a window wrapper for wasm based on stdweb https://github.com/koute/stdweb. I use it for OrbTk https://gitlab.redox-os.org/redox-os/orbtk. Maybe I could help to implement wasm support.

[tversteeg]

I've made a small attempt to set this up but I got stuck quite quickly. It seems you are well versed in WASM, maybe we could implement this feature together?

[FloVanGH]

@tversteeg sure why not. Do you use a specific library? I uses stdweb (https://github.com/koute/stdweb), it's very comfortable for interacting with web api on Rust wasm. You could find my wasm window shell wrapper on https://gitlab.redox-os.org/redox-os/orbtk/blob/master/crates/shell/src/web/mod.rs. The code could need a little bit refactoring and there are also OrbTk specific parts, but we could use it as example.

I also create a tool called cargo-node https://github.com/FloVanGH/cargo-node to build, deploy and run rust wasm applications as electron or cordova app. It could help us to test the port and we could use it also for a initial android port until there is a native port.

At the moment I have a lot of work todo on OrbTk but I can spent some time in a few weeks. I hope after the port is finished I can replace my wasm window shell wrapper.

[tversteeg]

I've created a small start here: #92 It's compiling but not working at all at the moment, I'm not sure what I'm doing wrong.

[FloVanGH]

@tversteeg do you tried to run it with cargo web? https://github.com/koute/cargo-web. I've tried it and get this output.

rror[E0433]: failed to resolve: use of undeclared type or module `imp`
   --> src/lib.rs:128:19
    |
128 | pub struct Window(imp::Window);
    |                   ^^^ use of undeclared type or module `imp`
error[E0433]: failed to resolve: use of undeclared type or module `imp`
   --> src/lib.rs:190:9
    |
190 |         imp::Window::new(name, width, height, opts).map(Window)
    |         ^^^ use of undeclared type or module `imp`
error[E0433]: failed to resolve: use of undeclared type or module `imp`
   --> src/lib.rs:682:17
    |
682 | pub struct Menu(imp::Menu);
    |                 ^^^ use of undeclared type or module `imp`
error[E0433]: failed to resolve: use of undeclared type or module `imp`
   --> src/lib.rs:695:9
    |
695 |         imp::Menu::new(name).map(Menu)
    |         ^^^ use of undeclared type or module `imp`
error: aborting due to 4 previous errors
For more information about this error, try `rustc --explain E0433`.
error: could not compile `minifb`.

To learn more, run the command again with --verbose.
error: build failed
	static/index.html

cargo-web start.
warning: unused manifest key: package.maintenance
   Compiling minifb v0.13.0 (/Users/florian.blasius/Downloads/rust_minifb)
error[E0433]: failed to resolve: use of undeclared type or module `imp`
   --> src/lib.rs:128:19
    |
128 | pub struct Window(imp::Window);
    |                   ^^^ use of undeclared type or module `imp`
error[E0433]: failed to resolve: use of undeclared type or module `imp`
   --> src/lib.rs:190:9
    |
190 |         imp::Window::new(name, width, height, opts).map(Window)
    |         ^^^ use of undeclared type or module `imp`
error[E0433]: failed to resolve: use of undeclared type or module `imp`
   --> src/lib.rs:682:17
    |
682 | pub struct Menu(imp::Menu);
    |                 ^^^ use of undeclared type or module `imp`
error[E0433]: failed to resolve: use of undeclared type or module `imp`
   --> src/lib.rs:695:9
    |
695 |         imp::Menu::new(name).map(Menu)
    |         ^^^ use of undeclared type or module `imp`
error: aborting due to 4 previous errors
For more information about this error, try `rustc --explain E0433`.
error: could not compile `minifb`.

[tversteeg]

Are you running the wasm branch? Because on master I get the same errors as you do.

[tversteeg]

I just read this article about using I/O devices in WASM. It would make adding a WASM target a lot cleaner if this would get into the official spec, another render loop with a callback wouldn't be necessary anymore. If it does happen I'll re-open the PR I made.

[dc740]

Hi, this ticket has been on hold for quite some time. What's the status? I took a quick glance at other implementations, and it seems like using the asynchronous .request_animation_frame from web-sys is the way to go, but I wonder how that plays with a simple loop like we use in minifb.

For reference, I took the time to remove dependencies on a demo game I found that uses exactly that, and it's been a real success. I was able to run the entire game in 100% rust, with only some glue javascript to load the wasm binary. The good part is that the javascript it uses is autogenerated, ¡and you don't need to use nor install npm!

https://emilio-moretti.medium.com/using-rust-and-webassembly-without-javascript-9058b8fb6a28 https://github.com/dc740/blobs-and-bullets

The game, and the explanation on how it works can be found here: https://rhmoller.dev/posts/2020/my-experience-with-rust-and-websys/

I was thinking about taking this task by myself, but I can't seem to find a way to abstract web-sys to make it look simple. minifb has something others don't: you can look at the implementation and figure what's going on or how to use it almost instantly. There are no puzzles on layers and layers of abstractions that tend to bloat most projects that want to be super generic. minifb is good at doing a few things, and it shines on those things for the simplicity. I wanted to replicate that experience with websys, but I simply can wrap my head around it.

[emoon]

Hi, So there has been some work on it it here https://github.com/emoon/rust_minifb/pull/92 but from what I understand is that it's hard to get it working with WASM the way minifb is currently structured in terms of event handling.

minifb has something others don't: you can look at the implementation and figure what's going on or how to use it almost instantly. There are no puzzles on layers and layers of abstractions that tend to bloat most projects that want to be super generic. minifb is good at doing a few things, and it shines on those things for the simplicity.

Thanks! This is something that is important to me and I try to keep the code straightforward :) Even if it has got a bit more complex over the years, I think it's still fairly easy to follow.

[dc740]

thank you very much. I share your philosophy. Sometimes we need complex code, and other times, we need code that is easy to maintain and has a low barrier to newcomers. I like to share knowledge, so I try to go with the later when possible. Specially on Rust, which already has a high learning curve due to the way it handles memory.

I took that PR, removed all style changes, and squashed the commits into one (found in my fork, under the wasm branch)

I'm glad to help here and I think we have two options:

1. Copy the arduino style: have a setup, loop and teardown functions that get called.
2. Create a thread where the main loop gets detached from the wasm related functionality.
3. Create a thread for the game loop and send a closure with it as explained above

I have not tried 1 and 2, but I think they will look much simpler to the end user. I like 1 for the simplicity of the backend implementation, and I also think it gets bonus points for being familiar to Arduino developers, which many times are beginners who are learning to code. Options 2 and 3 require threads, which are now mature and available, but... you increase the complexity of the implementation.

Speaking of implementations, I don't know how I would exit the game loop. Probably register the function, and stop the game loop if the function returns false, or something like that. So the actual control is still in the game loop, and not in the library implementation. I still want to be able to ask if the window is open or a key is pressed to decide if I want to quite the game loop. Specially for demos and short examples for students.

[emoon]

Personally I would (if possible) to keep the current flow like:

    let window = window::create(..);
    while window.is_open() {
      // do stuff
   }

Now, if this doesn't work for WASM perhaps we can have something optional for people targeting wasm and still keep the "old" style API around. I would prefer it worked as similar as possible, so if you want to target wasm the style (that is required in that case) also works with the current Linux, macOS, Windows backends and in those cases it will just be calling the regular API while being a bit hidden from the user.

This way we wouldn't break the current API compatibility and allow users to switch to the alternative API if they want to transition.

I also think that threads shouldn't be a requirement to use minifb as you pointed to as well.

[tversteeg]

These are the exact same problems I've encountered when experimenting with a solution in #92, I don't think that it will be possible to maintain @emoon's blocking while loop. The reason for this is that all the pixels need to be copied from the JavaScript context to the Rust context and back every frame. With my experiments it wasn't possible to do this ~50 times in a second. Even with threads this operation takes too long.

So I think the only solution would be to wait and hope that IO devices will be supported and that the JS pixel buffer can be shared with Rust without much overhead.

[emoon]

Right. I'm also open to an alternative update loop as written above, but I want the current one to be working as is. Also the new updated style needs to support the current backends.

So if you have some suggestions for this I'm all ears.

[tversteeg]

Even with an alternative update loop it will still be very challenging to achieve, because the performance problem of moving the buffers between the contexts still stands. And since there's no thread support in WASM yet the problems are exactly the same.

What I didn't research and what might be possible is that somehow JavaScript accesses the minifb buffer that's allocated in the WASM context directly and converts it to the pixels it can use. I'm assuming the performance of this is worse though. I don't give this a high probability of success though.

So my suggestion is, like I said, to wait and hope that IO devices will be supported and that the JS pixel buffer can be shared with Rust without much overhead.

[emoon]

I see. I haven't done any WASM stuff, but I (naively) figured it would be fast to move data in/out to JS.

[dc740]

I think you are right. it may or may not give a good performance, but I still think it's worth adding support for working on the HTMLCanvas from Rust like the projects I posted above. IMHO the problem of performance should be fixed by wasm-bindgen in the glue javascript code some day, and we shouldn't think too much on it. We could add a flag to use hardware acceleration when/if that becomes a standard and someone requires it, but I wouldn't stop at the performance problems right now. For me, getting basic support, and a simple api can be a good thing, even if its slow.

I personally would define an extra API. Something simple to work like an async api. So people who need webassembly could use minifb with the async api (enabling the flag manually!). This way we could keep the loop as it is right now, and offer an option that is webassembly friendly. Hey, it would even be async friendly, for those who prefer it.

[emoon]

It this is possible it sounds good to. If you would like to take a stab at it that would be great, just make sure to verify that an async version would work with existing backends also. This is so people targeting wasm can have the same code running for native as well.

[dc740]

Sorry I took so long. I was working on a candidate implementation. At first I was wondering what to do with the fact that the web does not like blocking code and then it hit me: I must not do anything.

I literally ignored the problem, since the developer who wants to build for the web will already be thinking in async code, so minifb job is not to get in the middle. This is the draft PR. I will work on the example over the week. Most likely on the weekend. https://github.com/emoon/rust_minifb/pull/239

Here is another demo if anyone wants to take a look or test the code (which I haven't done yet!): https://github.com/extraymond/rust-async-wasm-demo

The idea is to actually spawn a task and just keep updating the canvas from request_animation_frame, which runs more than 50 times a second. I wouldn't trust the timing there, but it took about 6 seconds to count from 0 to 300 in both, a 12 year old intel processor, and on a ryzen 5800. https://rustwasm.github.io/wasm-bindgen/examples/request-animation-frame.html

[dc740]

The PR is still a little rough around the edges, but I finally got a working demo: https://github.com/dc740/minifb-async-examples

it's flashing the screen on each animation frame. I need to add a different main to use the same app with other minifb backends, but you get the idea. minifb only provides the abstraction, and the developer is in charge of the main loop. You can choose to spawn_local, request_animation_frame, or even use other executors on a desktop app.

It's still a work in progress, but at least it works. Next steps: Improve the PR and improve the demo to run on other non-web backends.

[emoon]

Would it be possible to try to not use the RefCell stuff? I know that sometimes you have to use it, but if some design that doesn't require it would be great.

[dc740]

From the demo I posted last week, I think you can use spawn_local, and then setTimeout to render at your desired speed (let's say 30FPS). More info: https://forum.unity.com/threads/rendering-without-using-requestanimationframe-for-the-main-loop.373331/#post-5019212

I took the sample code from the official book: https://rustwasm.github.io/wasm-bindgen/examples/request-animation-frame.html

[dc740]

Just in case, the demo was not working. I didn't copy some files. It's fixed now, and it shows a bouncing green box.

[dc740]

OK, the PR is not ready, because there is no keyboard support right now (or at least I didn't try it). I had to extra long weeks, but I finally got the time to finish the demo: https://github.com/dc740/minifb-async-examples

The same code compiles in desktop AND web! It uses a regular loop for desktop, literally copy/paste from the minifb README, and it uses async request_anim_frame on the web.

As I said before, you could use spawn_local to execute a function many times, and only refresh at the desired FPS. I didn't try it. What I did try was the request_anim_frame FPS, and it looks like it's limited by the browser or vsync, I'm not sure. Cause I couldn't get more than ~50FPS, even on a ryzen 7 5800x with a 4gb video card. Tried X server and wayland, and both seem to update above 50FPS, even though the processors and cards are completely different. (the X server was running on an intel 7400 with 256mb ati radeon express).

The great thing about getting some basic support, is that you can later add webgl, and get a nice boost. I didn't follow this because my old laptop can't run webgl, so I'm interested on having an alternative. Regarding your previous request: I'm not sure I can avoid refcell, I haven't looked into it, since I'm a little short on time lately.

[dc740]

I have not been able to look at this as much as I'd like. I'll leave the current status detailed, just in case anyone wants to help (much needed):

This is the working branch: https://github.com/dc740/rust_minifb/tree/wasm_websys_wip

Status:

• It works!
• You can create a project that uses the same code for web and desktop (see the demo I published)
• It runs at the monitor sync rate (this is how request anim frame works, but check the TODO list)
• There is no need to set the rate, since there is no blocking code
• It get keys and mouse events so you can move the demo box with the arrows.
• all keyboard keys are mapped
• mouse is mapped

BUGS:

• None that I've found, but there are a couple of unimplemented features

TODO and "nice to have":

• Clean up the Instant requirement. I think it's only needed for the web part, so it doesn't make sense to use it globally (although it changes nothing)
• Add support to resize the canvas HTML element. Right now it simply spans all the visible space.
• modify the demo so the game is run with spawn_local, and leave the final buffer copy in the request_animation_frame
• evaluate Shared Array Buffer (now it's supported again!!!) combined with web workers https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer
• evaluate OffscreenCanvas https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas
• if the last two items work OK, we can finally have a blocking loop in a web worker and we can share even more code. Right now the project needs to be setup as I show here: https://github.com/dc740/minifb-async-examples

[dc740]

Just a heads up, it's merged!

Of course it can be improved but you can run minifb on the web now. I cleaned up the multiplatform demo a little bit: https://github.com/dc740/minifb-async-examples

The same code compiles to desktop, and there is a script to compile to wasm32 and create a web page to load it.

[emoon]

Closing as (initial) support has landed