Asynchronous WASM Compilation as an option for NodeJS Targets

[Nyrox]

Related to #50 and #51 .

I am bringing this old basket back up, since I'd like to ask if there is any interest in supporting this feature straight out of the box. Chromium still doesn't allow compilation of WASM modules over 4KB on the main thread, which is particularily troublesome for using wasm-bindgen for developing electron applications. To use wasm in electron as far as I can tell you currently still need to use more external tooling, uneccsarily complicating the build chain and raising the bar for getting started with electron and wasm.

I'd propose being able to switch between a synchronous and asynchronous promise based loader with a CLI flag. If there is shared interest for it, I'd be happy to try and implement it, but no promises on my ability to do so, pun intended.

[alexcrichton]

Thanks for the report! I'm personally pretty unfamiliar with electron so I'm hoping you can help me at least understand a bit more? How does electron and node.js relate? Are you executing in a browser with node-like APIs? For example, how come you wouldn't use the browser target with electron?

It certainly wouldn't be too hard to add this, but I'd just want to be sure to confirm what's not working today and how we can work best with electron!

[Nyrox]

Well basically Electron gives you a Browser front end which also supports the entire NodeJS API set, including loading NodeJS modules on the client side, which in my limited experience would be the preferred way of running WASM code on the render thread. Currently this fails though, because the NodeJS target compiles using the blocking functions in the WebAssembly API and Chromium (the front end Electron uses internally) doesn't allow using the blocking functions on the main thread for Modules over 4K.

You bring up a good point in that you could technically use the Browser Target and it would work with the "hack" introduced in #51 for Chrome, but I'd like to make this a stylistic choice and not a forced one, because many projects in electron are managed using NodeJS modules rather than tools like Webpack.

Also this would basically allow you to take whatever wasm-pack spits out and straight up require it in your code, which would be a benefit in itself for introducing rust into living codebases imo. 😃

[alexcrichton]

Ok cool, thanks for clarifying!

Unfortunately though I'm not sure how well an asynchronous compile would work. It sounds like management of modules in Electron are done through native NodeJS packages, which today use synchronous compilation of wasm because Node allows it. If we were to switch to asynchronous compilation, however, this means that the require of the Node module which has wasm wouldn't actually be ready until a later time (when the compile finishes). This in turn means that anything which expects the required package to be immediately available will break.

Do you know if there's a way to thread the promise from asynchronous compilation into the system? Is there a good way to block on that promise before making progress?

Conversely since Electron looks to be pretty custom in this respect, could Electron be updated to remove the 4k limit for wasm modules and synchronous compilation?

[roccomuso]

I'm in favor of this. Async should be configurable. Having this issue too (https://github.com/rustwasm/create-wasm-app/issues/67).

@alexcrichton electron cannot be updated to remove this 4k limit, it inherits this from chrome itself and there's no will to get rid of this limit.

[sunjay]

@alexcrichton What do you think of having a nodejs-async target (or something)? That way the nodejs target would remain normal but modules could still be used from electron within the 4k limit.

[alexcrichton]

I agree yeah the best way to support this will likely be a new target, although apart from asynchronous initialization I'm not entirely certain what this target would look like (although probably similar to the web target)

[tylerthecoder]

Is there any update on this?

[Nyrox]

Not as far as I know. I did have a look around yesterday to see about the state of things, also because when I opened this issue I didn't really understand the different JS module standards (still barely do, but now I have to pretend to for work).

Sadly what I thought would be the best method available, which is using just straight up ES modules the whole way, which is starting to be more and more feasible, is not possible in electron still, and I have to admit I am a bit salty at the electron devs for it. They seem adamant about not allowing the entrypoint to a render thread to be a ES module and any workarounds are pretty gnarly.

Now that aside, I would have to look at the exact output of the different backends to decide which one makes the most sense, but I think it should probably be possible to just use tsc and use one of them without a full bundler. But if not I guess we are still stuck with just using webpack for our electron apps. 😭

Edit: To clarify on the last part: basically I think you should probably be using commonJS modules instead of require for your electron code as well. That is what I have been doing on my nodejs projects for years and it should allow you to use either the bundler or the web backend with just tsc with the right settings, if I am not mistaken.