WASI support

[daxpedda]

This issue will be used to collect discussions and issues about WASI support in wasm-bindgen.

Generally speaking wasm-bindgen doesn't currently support WASI, but we should figure out if we want to support it (https://github.com/rustwasm/wasm-bindgen/pull/3324) or disable it (https://github.com/rustwasm/wasm-bindgen/pull/3233).

The original plan, outlined here: https://github.com/rustwasm/wasm-bindgen/issues/1841, now doesn't have the necessary developer support. But the motivation for it, the component model (previously interface types), isn't here yet either.

Input from stakeholders would help a lot!

[csnover]

A library I was working with compiled and linked to a C library using the cc crate, and I believe that wasm-bindgen is not usable in that case unless it supports WASI. (I say ‘believe’ because I was never able to find a channel that was still active to get confirmation from someone authoritative, but at least it seemed like WASI was the ‘blessed’ way to make this work. I at least was unable to find some other way to make it work in the time I had to investigate.)

[daxpedda]

I guess this is one use-case: having WASI dependencies.

But I would argue that this is really a bad workaround, C libraries can target Wasm without WASI or Emscripten just fine. If any WASI function is used it wouldn't work in the browser anyway without a compatibility shim.

On the other hand I understand that the only Wasm target C/C++ libraries usually support is Emscripten, or WASI if they are more modern. As hacky as I described this above, it's a workaround many users could require.

Though I've honestly no clue if you can use a WASI dependency successfully or not with wasm-bindgen, so more input would be appreciated. Thanks @csnover.

[csnover]

Getting a WASI provider into the browser that covered the required system calls was trivial with https://github.com/bjorn3/browser_wasi_shim and required less than 10 LOC change to the bootstrap JS to make it send the required imports when instantiating the module, so this is no barrier.

The third-party C dependency has no knowledge of wasm and used system calls which are available in the WASI shim; I do not understand the comment about it being a ‘bad workaround’ so would be interested to understand more where you are coming from in this regard.

I am not paying attention to the wasm ecosystem to understand what is what, but as an outsider exploring the space for the first time, it looked like WASI being positioned as the standard ABI. Maybe that is just an wrong impression though?

[daxpedda]

Getting a WASI provider into the browser that covered the required system calls was trivial with https://github.com/bjorn3/browser_wasi_shim and required less than 10 LOC change to the bootstrap JS to make it send the required imports when instantiating the module, so this is no barrier.

I agree, it's not a barrier.

The third-party C dependency has no knowledge of wasm and used system calls which are available in the WASI shim; I do not understand the comment about it being a ‘bad workaround’ so would be interested to understand more where you are coming from in this regard.

So I'm searching for the "how it's supposed to be done". WASI isn't supported by the browser, that's why a shim is required.

The "proper" way to do it is to compile to Wasm, not to WASI, which C/C++ is able to. Any C/C++ library not using any system calls will support the Wasm target (barring any complications with build systems :rofl:), but as mentioned above, C/C++ libraries that do use system calls, don't commonly support targeting Wasm, but Emscripten.

(I'm using the term "Wasm" here as in the bare target, without WASI or Emscripten.)

---

I'm not against supporting WASI in wasm-bindgen by the way. Just trying to determine how useful, necessary and burdensome it will be.

The original plan for wasm-bindgen was to support WASI anyway, but in a very different way we are now not able to do anymore, see https://github.com/rustwasm/wasm-bindgen/issues/1841.

[csnover]

I understand what you were saying now, thank you for sharing.

For what it is worth I don’t have much skin in the game at all here; I was just told by someone recently that the emscripten target for Rust did not work, took that claim at face value, saw that WASI was being developed under the WebAssembly group, and assumed this meant the ecosystem had agreed to move toward WASI as the universal ABI for runtimes that need system interface.

I was able to hack wasm-bindgen into working with the wasi target within a day so it seems like it’s mostly there, but it is possible that I didn’t exercise the interface enough since I was only doing it as a proof of concept.

Today I read some discussions in the WASI WG repo and have much more uncertainty as to whether there is actually a commitment to ensure first-class web platform support. Hopefully they are able to resolve that one way or the other since this is the sort of ambiguity that kills standards.

Let me know if there is any other feedback I can provide. Thanks again!

[itsezc]

Not at all sure if its entirely related but as someone who actively uses the web-sys crate, I'd love to see component model come to life (sooner than later) and hope that application developers can get away with little to no refactoring (just a dream); I'd love to see WASI supported.

[gbj]

As the one who proposed #3233 and as the maintainer of a now-large-ish Rust/WASM web framework: I am fine with any solution that is an actual solution.

At present, wasm-bindgen generates bindings that panic if called outside the browser. This is completely fine for me: it means that we can use types like web_sys::Event (and other things that depend on wasm-bindgen) to define event handlers etc. as long as they aren't actually called on the server, in which case they panic. Any framework is perfectly capable of handling this and simply not calling code that will panic in this way.

However, in current wasm32-wasi runtimes server-side compiling with a wasm-bindgen tends to lead to a runtime panic due to unknown exports even if they are never called. This means that wasm32-wasi behaves quite differently from any non-WASM server target.

For additional context see the following

https://github.com/leptos-rs/leptos/issues/295

https://github.com/bytecodealliance/wasmtime/issues/3937

https://github.com/bytecodealliance/wasmtime/issues/5557

[temeddix]

A library I was working with compiled and linked to a C library using the cc crate, and I believe that wasm-bindgen is not usable in that case unless it supports WASI. (I say ‘believe’ because I was never able to find a channel that was still active to get confirmation from someone authoritative, but at least it seemed like WASI was the ‘blessed’ way to make this work. I at least was unable to find some other way to make it work in the time I had to investigate.)

As @csnover mentioned, wasm32-unknown-unknown has too much limitations that it won't work well on web browsers. Major wasm runtimes and major crates including tokio now support WASI, but it's only wasm-bindgen that does not. If we get to support WASI, so many crates that depend on system time and threads will be able to work on web browsers, with the power of WASI polyfills like wasmer-js, and that would indeed be wonderful.

Since wasm-bindgen is the de-facto standard of Rust ecosystem for wasm, IMHO extending support to wasm32-wasi would benefit so many users out there working for the web.

[temeddix]

#3454 seems to be the initial progress toward WASI goal, so I'll leave the link here.

• https://github.com/rustwasm/wasm-bindgen/issues/3454

[daxpedda]

As @csnover mentioned, wasm32-unknown-unknown has too much limitations that it won't work well on web browsers.

wasm32-unknown-unknown is not a browser target, which is why it is so limited, but wasm32-wasi isn't either. Hopefully, at some point in the future, a new wasm32-web target can be introduced. This would require the component model though.

Major wasm runtimes and major crates including tokio now support WASI, but it's only wasm-bindgen that does not.

I think in general Rust projects can add support for wasm32-unknown-unknown. Tokio for example, wouldn't work in the browser, AFAIU, even if wasm-bindgen supports wasm32-wasi, see https://github.com/tokio-rs/tokio/issues/5418.

---

I generally don't think that WASI is entirely out of scope for wasm-bindgen. But supporting WASI just to work around issues of getting Rust crates working in the browser, is imo a mistake that will only lead to confusion. wasm32-wasi is not a browser target and probably never will be.

Adding the WASI shim though is something that makes sense to me, because unlike Rust, C/C++ support for the browser is not "really" a thing (except through Emscripten). Rust libraries can use wasm-bindgen, C/C++ can not.

So the way forward here to me is:

1. Add browser support to Rust libraries that need it by using wasm-bindgen for the wasm32-unknown-unknown target. This has worked wonderfully until now, see Winit or Wgpu for example.
2. Find a Rust library that cannot support wasm32-unknown-unknown but can support wasm32-wasi so we can discuss why wasm-bindgen actually needs to support wasm32-wasi. E.g. Tokio is not one of those, it's WASI implementation doesn't work in the browser.

It is obviously very convenient to just slap WASI support onto wasm-bindgen and call it a day, but the point I'm trying to make here is that this is a tradeoff, there are downsides to this. E.g. using Tokio in the browser only to find out that it either will just crash, because it blocks, or if running in a web worker, just never yield back to the browser runtime, making e.g. events never trigger.

I hope this clarifies why I believe that the "wasm32-unknown-unknown target is too limited" argument doesn't make a whole lot of sense to me.

[temeddix]

wasm32-wasi is not a browser target and probably never will be.

Agreed, I was just trying to point out that WASI can run on the browser via polyfill libraries. The main benefit would be std almost fully working just like native, though specific behavior on the web will depend on polyfill libraries(Such as showing the folder dialog to user to access user folder with JS FileSystemDirectoryHandle).

I think in general Rust projects can add support for wasm32-unknown-unknown.

Yeah, this is how the Rust web ecosystem is evolving right now. However, many crates won't work unless each of them implements a solution for wasm that bypasses std by interacting with JavaScript. Simply being able to depend on std would be a huge benefit.

E.g. using Tokio in the browser only to find out that it either will just crash, because it blocks, or if running in a web worker, just never yield back to the browser runtime, making e.g. events never trigger.

Yes, blocking the browser's main JS runtime should never happen. But since atomics are already available in wasm, access to system time and threads via std will possibly make tokio on the web work again. I was looking forward to creating a PR at tokio repo that yields periodically to the JS runtime(Pehaps every 1ms, or even less) once WASI can run in browsers. We can already use wasm_bindgen_futures to spawn Futures in JS runtime, but this doesn't allow us to use extensive tokio features such as task_local!. While it's true that tokio's current WASI implementation doesn't work on the web, a simple PR that makes the runtime yield periodically can do the trick.

Find a Rust library that cannot support wasm32-unknown-unknown but can support wasm32-wasi so we can discuss why wasm-bindgen actually needs to support wasm32-wasi.

Yes, this would be a nice research to make. Thanks for the idea.

[daxpedda]

While it's true that tokio's current WASI implementation doesn't work on the web, a simple PR that makes the runtime yield periodically can do the trick.

Yes, but this can't be added to wasm32-wasi, this should be added as wasm32-unknown-unknown support to Tokio.

I was looking forward to creating a PR at tokio repo that yields periodically to the JS runtime(Pehaps every 1ms, or even less) once WASI can run in browsers.

I would argue that this is pretty crude and probably won't be accepted by Tokio. To properly add support for wasm32-unknown-unknown, I think a completely new executor would be required.

See https://github.com/WICG/scheduling-apis/blob/main/explainers/yield-and-continuation.md for proper yield functionality in JS, but even that I believe is not the way to move forward in Tokio.

[temeddix]

There was an opinion at the forum about this WASI support and ABI mismatch.

• https://users.rust-lang.org/t/fixing-rusts-webassembly-targets/88947

[temeddix]

I saw that #3673 was just closed because directly supporting wasm32-wasi target 'looks like a mistake'.

• #3673

But.. I don't really understand why wasm32-wasi shouldn't be supported from wasm-bindgen directly, not even as experimental.

Add browser support to Rust libraries that need it by using wasm-bindgen for the wasm32-unknown-unknown target. This has worked wonderfully until now, see Winit or Wgpu for example.

Honestly, I don't think this is the ideal way, because this is what causes fragmentation in the Rust's web ecosystem, forcing crates to write the code twice for wasm and non-wasm versions. Although this works, it doesn't look elegant.

Hopefully, at some point in the future, a new wasm32-web target can be introduced.

This would be elegant, but then should we add this new target with a PR at Rust? If so, who should? It seems a person like me who doesn't have proper permissions at wasm-bindgen cannot suggest such things.. I can see that this would take many more years at the current speed of wasm infrastructure development. What I've seen much in this wasm related community is 'in the near future', and 'someday' being repeated for 3 years, with nobody taking the move...

I personally believe that wasm32-wasi can be deno/nodejs/browser target, if combined with proper JavaScript shims. Would listing crates that would benefit from wasm-bindgen supporting wasm32-wasi be enough to persuade?

Hope @hamza1311 , @LachezarLechev can share their opinions as well. However, if this whole idea cannot be accepted, I will be just silented for good..

[daxpedda]

@temeddix I don't want to discourage you from discussing this or voicing your opinion, it is in fact very much appreciated. But this needs further discussion and I am aware that discussing things in this context is work as well, it requires care, research and effort.

I believe I have outlined my reasons here https://github.com/rustwasm/wasm-bindgen/issues/3421#issuecomment-1780741778 as clear as I can. Specifically we have continued discussion with the example of Tokio support, which as far as I am aware will not be solved by adding WASI support to wasm-bindgen, hence another reason why I'm so far of the opinion that this isn't the best way forward.

In addition to what I said in #3421, I think the path forward right now to improve Rust browser compatibility:

• Solve #3454, so can enable support for C/C++ dependencies.
• Work on stabilizing features like "atomics".

If anything is unclear please let me know.

[daxpedda]

Add browser support to Rust libraries that need it by using wasm-bindgen for the wasm32-unknown-unknown target. This has worked wonderfully until now, see Winit or Wgpu for example.

Honestly, I don't think this is the ideal way, because this is what causes fragmentation in the Rust's web ecosystem, forcing crates to write the code twice for wasm and non-wasm versions. Although this works, it doesn't look elegant.

I agree that this isn't elegant. But as I pointed out, there is no guarantee that things would just work. E.g., your own example, Tokio.

Hopefully, at some point in the future, a new wasm32-web target can be introduced.

This would be elegant, but then should we add this new target with a PR at Rust? If so, who should? It seems a person like me who doesn't have proper permissions at wasm-bindgen cannot suggest such things..

Here we are, you are suggesting things, and we are discussing them.

I can see that this would take many more years at the current speed of wasm infrastructure development. What I've seen much in this wasm related community is 'in the near future', and 'someday' being repeated for 3 years, with nobody taking the move...

You could be that somebody!

Would listing crates that would benefit from wasm-bindgen supporting wasm32-wasi be enough to persuade?

It would help continuing the discussion.

---

I've read your post after I wrote https://github.com/rustwasm/wasm-bindgen/issues/3421#issuecomment-1785876748, but let me re-iterate: I really really encourage you to have a discussion about the solution to this problem. Your frustration on the state of things is understandable and shared. But slapping quick fixes left and right isn't imo the way to move forward here. In fact I believe this would add to the fragmentation, not reduce it. This problem is not easy, it is complex, it requires discussion and design.

@temeddix I'm not even sure what you are hoping to fix exactly with adding wasm32-wasi support. The only thing you mentioned is Tokio, which as we discussed, wouldn't be fixed by that. std::time could be made to work, but is there something blocking users from using web-time or instant? Is there a library you know which doesn't want to support wasm32-unknown-unknown but is willing to support WASI? Do you disagree with the arguments we made here why adding WASI support directly is a bad idea? If yes, could you elaborate?

[temeddix]

Thanks for the clarification.

I do disagree with the arguments that adding WASI support directly is a bad idea. Indeed, there's nothing blocking users from using wasm helpers like web-time, but this is not really an elegant solution and I believe Rust ecosystem for the web shouldn't go that way. For example, they would have to write this...

#[cfg(target_family="wasm")]
use web_time::Instant;
#[cfg(not(target_family="wasm"))]
use std::time::Instant;

fn main() {
    let now = Instant::now();
}

instead of this

use std::time::Instant;

fn main() {
    let now = Instant::now();
}

This was just a simple case, but in large codebases it really gets complex. Also, we need to do additional research about whether web_time supports all the std::time functionalities or not. Many wasm crates claim that they aim to support certain std feature as a drop-in replacement, but there are missing pieces here and there. This is why I suggested that being able to fully use std on web browsers should be allowed with wasm32-wasi.

I will try to elaborate on my points later on.

[vmx]

2. Find a Rust library that cannot support wasm32-unknown-unknown but can support wasm32-wasi so we can discuss why wasm-bindgen actually needs to support wasm32-wasi. E.g. Tokio is not one of those, it's WASI implementation doesn't work in the browser.

@daxpedda an example is https://github.com/rustpq/pqcrypto. It uses a C library that needs randomness. With wasm32-wasi it's calling arc4random_buf(), which isn't available on wasm32-unknown-unknown.

[CryZe]

I don't see an inherent reason why that function couldn't exist on wasm32-unknown-unknown.

[vmx]

I don't see an inherent reason why that function couldn't exist on wasm32-unknown-unknown.

I would expect it to be similar problem as for the WebAssembly support in the getrandom crate.

[CryZe]

That's not really a "problem" as much as it's the "state of the art". The entire ecosystem works that way. At least until there's a separate -web target.

[temeddix]

To me, that looks like a 'problem' because unless a crate aims to support webassembly it doesn't work on the web at all. They have to write additional code or add additional feature to make it work on the web.

With wasm32-wasi, this code duplication is not needed, mainly because it can use std and the standard C ABI.

[daxpedda]

I do disagree with the arguments that adding WASI support directly is a bad idea.

I will try to elaborate on my points later on.

Please do, this is what we need to do here!

For example, they would have to write this...

Actually you can just write:

use web_time::Instant;

fn main() {
    let now = Instant::now();
}

... which works correctly on all platforms.

---

@daxpedda an example is https://github.com/rustpq/pqcrypto. It uses a C library that needs randomness. With wasm32-wasi it's calling arc4random_buf(), which isn't available on wasm32-unknown-unknown.

This is the kind of stuff we do want to support. A linked library can be compiled to WASI to support C/C++ dependencies which can't use wasm-bindgen, but Rust can stay wasm32-unknown-unknown. Though I don't exactly know what arc4random_buf() is, but I assume it relies on WASI random? If it's not something supported by WASI, then I'm not sure what the point here is.

---

To me, that looks like a 'problem' because unless a crate aims to support webassembly it doesn't work on the web at all. They have to write additional code or add additional feature to make it work on the web.

With wasm32-wasi, this code duplication is not needed, mainly because it can use std and the standard C ABI.

I believe this is the crux of the problem here and I'm assuming where you were coming from this whole time.

This is what we ought to discuss here. As far as I've experienced it, slapping WASI support on top will cause issues exactly for this reason.

E.g. Tokio now has WASI support, but doesn't work in the browser. If Tokio wants to add browser support, they can't add it to wasm32-wasi, because that would break WASI support, they have to use a different target, e.g. wasm32-unknown-unknown. So if some of the ecosystem supports the browser through wasm32-unknown-unknown and some of the ecosystem through wasm32-wasi, we will have target incompatibility.

@temeddix I encourage you to get into the specifics here. Which crates did you have problems with? What part of the ecosystem lacks browser support that you would like to see supported? How exactly could WASI support be applied to those crates to add browser support without creating target incompatibility?

[temeddix]

I will organize the list of crates that has issues on wasm32-unknown-unknown soon, after I have my daily tasks done.

Also, I get the point that wasm32-wasi is not for browsers. However, since wasm32-unknown-unknown is also not for browsers, what do you think about adding wasm32-wasi-browser or wasm32-wasi-js target to Rust, which is basically WASI but intended to run in browser JavaScript environment?

Some people do suggest deprecating wasm32-unknown-unknown:

• https://users.rust-lang.org/t/help-us-make-wasm-bindgen-support-more-target/101718

[daxpedda]

Also, I get the point that wasm32-wasi is not for browsers. However, since wasm32-unknown-unknown is also not for browsers, [..]

This is true! It shows too: this is why we are having all these problems here!

[..], what do you think about adding wasm32-wasi-browser or wasm32-wasi-js target to Rust, which is basically WASI but runs in browser Javascript environment?

I'm not a Rust maintainer/member, so you are asking the wrong person. But I doubt anything like that will be accepted by Rust.

The reason why wasm32-unknown-unknown is being used as a browser target is because we don't have a better choice right now, it's only an ad-hoc solution to a problem we can't solve any other way. But I'm not sure how exactly you are going to convince the Rust team that a wasm32-wasi-web target has a purpose if it's exactly the same as wasm32-wasi (or how exactly would it be different? after all JS support can't be added to rustc, unfortunately). Target incompatibility is "just" a downstream issue for rustc as long as an alternative exists: wasm32-unknown-unknown.

I'm not aware of any effort to standardize WASI in browsers, but if there would be, I think that might be an argument. But again, it has to differ somehow to wasm32-wasi.

But, to re-iterate: this is something you have to take up with the Rust team.

[daxpedda]

Some people do suggest deprecating wasm32-unknown-unknown:

• https://users.rust-lang.org/t/help-us-make-wasm-bindgen-support-more-target/101718

I doubt wasm32-unknown-unknown will ever be deprecated. But it is true that it has been misused so far as a browser target, which is very unfortunate.

wasm32-web will happen (:crossed_fingers:), but it requires the component model, I'm not aware of any other way to access browser APIs in rustc without it.

[temeddix]

If wasm32-wasi-js is accepted by Rust maintainers, would it be possible that wasm-bindgen extends support to that target?

Also, I'm wondering even if WASI component model is accepted someday, wouldn't wasm32-wasi-js still be there to allow using that component model with wasm-bindgen in JavaScript environment?

Lastly, maybe wasm32-wasi-js could be a clearer target description for all deno/nodejs/browser. What do you think? Because it's basically Rust in JavaScript environment. Or should it be wasm32-wasi-web?

If I have some consensus from wasm-bindgen side, then at least I can take this and suggest the new target at Rust side, though not sure if it will be accepted.

[daxpedda]

If wasm32-wasi-js is accepted by Rust maintainers, would it be possible that wasm-bindgen extends support to that target?

If it does exist, I don't see why we shouldn't support it in wasm-bindgen, wish I had more input though.

Also, I'm wondering even if WASI component model is accepted someday, wouldn't wasm32-wasi-js still be there to allow using that component model with wasm-bindgen in JavaScript environment?

I'm not aware of any WASI component model, the component model proposal is a Wasm proposal, not a WASI one. So I would say no. I also wouldn't know why anybody would still continue to use WASI in the browser when the component model is available and presents itself as a working alternative.

Lastly, maybe wasm32-wasi-js could be a clearer target description for all deno/nodejs/browser. What do you think? Because it's basically Rust in JavaScript environment. Or should it be wasm32-wasi-web?

I would vote for wasm32-wasi-web.

If I have some consensus from wasm-bindgen side, then at least I can take this and suggest the new target at Rust side, though not sure if it will be accepted.

Personally I would like to know first what this target is supposed to be exactly. But then sure, we can try to get some consensus here.

FWIW, my assumption here is that Rust doesn't (and imo shouldn't) care about wasm-bindgens role in all this anyway.

[vmx]

This is the kind of stuff we do want to support. A linked library can be compiled to WASI to support C/C++ dependencies which can't use wasm-bindgen, but Rust can stay wasm32-unknown-unknown.

@daxpedda Can I today compile a Rust crate as wasm32-wasi and link that with wasm32-unknown-unknown? If yes, pointers would be welcome.

Though I don't exactly know what arc4random_buf() is, but I assume it relies on WASI random? If it's not something supported by WASI, then I'm not sure what the point here is.

arc4random_buf() is just a call to a random number generator which is supported by WASI.

[daxpedda]

This is the kind of stuff we do want to support. A linked library can be compiled to WASI to support C/C++ dependencies which can't use wasm-bindgen, but Rust can stay wasm32-unknown-unknown.

@daxpedda Can I today compile a Rust crate as wasm32-wasi and link that with wasm32-unknown-unknown? If yes, pointers would be welcome.

Yes and no. It should be theoretically possible, though I wouldn't be able to give you any pointers on it as I've never tried it myself, but practically you will very likely hit #3454.